Apr 14 02:53:40 user nova-compute[71323]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
Apr 14 02:53:43 user nova-compute[71323]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' {{(pid=71323) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 02:53:43 user nova-compute[71323]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' {{(pid=71323) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 02:53:43 user nova-compute[71323]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' {{(pid=71323) initialize /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:44}}
Apr 14 02:53:43 user nova-compute[71323]: INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
Apr 14 02:53:43 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): grep -F node.session.scan /sbin/iscsiadm {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:53:43 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "grep -F node.session.scan /sbin/iscsiadm" returned: 0 in 0.019s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:53:43 user nova-compute[71323]: INFO nova.virt.driver [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Loading compute driver 'libvirt.LibvirtDriver'
Apr 14 02:53:44 user nova-compute[71323]: INFO nova.compute.provider_config [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access.
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Acquiring lock "singleton_lock" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Acquired lock "singleton_lock" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Releasing lock "singleton_lock" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Full set of CONF: {{(pid=71323) _wait_for_exit_or_signal /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:362}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ******************************************************************************** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Configuration options gathered from: {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] command line args: ['--config-file', '/etc/nova/nova-cpu.conf'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] config files: ['/etc/nova/nova-cpu.conf'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ================================================================================ {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] allow_resize_to_same_host      = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] arq_binding_timeout            = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] backdoor_port                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] backdoor_socket                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] block_device_allocate_retries  = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] block_device_allocate_retries_interval = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cert                           = self.pem {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute_driver                 = libvirt.LibvirtDriver {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute_monitors               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] config_dir                     = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] config_drive_format            = iso9660 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] config_file                    = ['/etc/nova/nova-cpu.conf'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] config_source                  = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] console_host                   = user {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] control_exchange               = nova {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cpu_allocation_ratio           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] daemon                         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] debug                          = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] default_access_ip_network_name = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] default_availability_zone      = nova {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] default_ephemeral_format       = ext4 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] default_schedule_zone          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] disk_allocation_ratio          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] enable_new_services            = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] enabled_apis                   = ['osapi_compute'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] enabled_ssl_apis               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] flat_injected                  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] force_config_drive             = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] force_raw_images               = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] graceful_shutdown_timeout      = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] heal_instance_info_cache_interval = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] host                           = user {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] initial_cpu_allocation_ratio   = 4.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] initial_disk_allocation_ratio  = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] initial_ram_allocation_ratio   = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] injected_network_template      = /opt/stack/nova/nova/virt/interfaces.template {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_build_timeout         = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_delete_interval       = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_format                = [instance: %(uuid)s]  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_name_template         = instance-%08x {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_usage_audit           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_usage_audit_period    = month {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] instances_path                 = /opt/stack/data/nova/instances {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] internal_service_availability_zone = internal {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] key                            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] live_migration_retry_count     = 30 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_config_append              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_dir                        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_file                       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_options                    = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_rotate_interval            = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_rotate_interval_type       = days {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] log_rotation_type              = none {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] long_rpc_timeout               = 1800 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_concurrent_builds          = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_concurrent_live_migrations = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_concurrent_snapshots       = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_local_block_devices        = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_logfile_count              = 30 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] max_logfile_size_mb            = 200 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] maximum_instance_delete_attempts = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metadata_listen                = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metadata_listen_port           = 8775 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metadata_workers               = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] migrate_max_retries            = -1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] mkisofs_cmd                    = genisoimage {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] my_block_storage_ip            = 10.0.0.210 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] my_ip                          = 10.0.0.210 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] network_allocate_retries       = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] osapi_compute_listen           = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] osapi_compute_listen_port      = 8774 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] osapi_compute_unique_server_name_scope =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] osapi_compute_workers          = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] password_length                = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] periodic_enable                = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] periodic_fuzzy_delay           = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] pointer_model                  = ps2mouse {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] preallocate_images             = none {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] publish_errors                 = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] pybasedir                      = /opt/stack/nova {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ram_allocation_ratio           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rate_limit_burst               = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rate_limit_except_level        = CRITICAL {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rate_limit_interval            = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reboot_timeout                 = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reclaim_instance_interval      = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] record                         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reimage_timeout_per_gb         = 20 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] report_interval                = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rescue_timeout                 = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reserved_host_cpus             = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reserved_host_disk_mb          = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reserved_host_memory_mb        = 512 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] reserved_huge_pages            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] resize_confirm_window          = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] resize_fs_using_block_device   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] resume_guests_state_on_host_boot = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rootwrap_config                = /etc/nova/rootwrap.conf {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rpc_response_timeout           = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] run_external_periodic_tasks    = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] running_deleted_instance_action = reap {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] running_deleted_instance_poll_interval = 1800 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] running_deleted_instance_timeout = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler_instance_sync_interval = 120 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_down_time              = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] servicegroup_driver            = db {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] shelved_offload_time           = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] shelved_poll_interval          = 3600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] shutdown_timeout               = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] source_is_ipv6                 = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ssl_only                       = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] state_path                     = /opt/stack/data/nova {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] sync_power_state_interval      = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] sync_power_state_pool_size     = 1000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] syslog_log_facility            = LOG_USER {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] tempdir                        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] timeout_nbd                    = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] transport_url                  = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] update_resources_interval      = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_cow_images                 = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_eventlog                   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_journal                    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_json                       = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_rootwrap_daemon            = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_stderr                     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] use_syslog                     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vcpu_pin_set                   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plugging_is_fatal          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plugging_timeout           = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] virt_mkfs                      = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] volume_usage_poll_interval     = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] watch_log_file                 = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] web                            = /usr/share/spice-html5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_concurrency.disable_process_locking = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_concurrency.lock_path     = /opt/stack/data/nova {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.auth_strategy              = keystone {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.compute_link_prefix        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.dhcp_domain                = novalocal {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.enable_instance_password   = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.glance_link_prefix         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.instance_list_per_project_cells = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.list_records_by_skipping_down_cells = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.local_metadata_per_cell    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.max_limit                  = 1000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.metadata_cache_expiration  = 15 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.neutron_default_tenant_id  = default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.use_forwarded_for          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.use_neutron_default_nets   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_dynamic_ssl_certfile =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_dynamic_targets = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_jsonfile_path   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api.vendordata_providers       = ['StaticJSON'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.backend                  = dogpile.cache.memcached {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.backend_argument         = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.config_prefix            = cache.oslo {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.dead_timeout             = 60.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.debug_cache_backend      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.enable_retry_client      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.enable_socket_keepalive  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.enabled                  = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.expiration_time          = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.hashclient_retry_attempts = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.hashclient_retry_delay   = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_dead_retry      = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_password        =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_pool_maxsize    = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_sasl_enabled    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.memcache_username        =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.proxies                  = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.retry_attempts           = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.retry_delay              = 0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.socket_keepalive_count   = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.socket_keepalive_idle    = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.socket_keepalive_interval = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.tls_allowed_ciphers      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.tls_cafile               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.tls_certfile             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.tls_enabled              = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cache.tls_keyfile              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.auth_section            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.auth_type               = password {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.cafile                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.catalog_info            = volumev3::publicURL {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.certfile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.collect_timing          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.cross_az_attach         = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.debug                   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.endpoint_template       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.http_retries            = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.insecure                = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.keyfile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.os_region_name          = RegionOne {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.split_loggers           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cinder.timeout                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.cpu_dedicated_set      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.cpu_shared_set         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.image_type_exclude_list = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.max_concurrent_disk_ops = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.max_disk_devices_to_attach = -1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.resource_provider_association_refresh = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.shutdown_retry_interval = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] compute.vmdk_allowed_types     = ['streamOptimized', 'monolithicSparse'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] conductor.workers              = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] console.allowed_origins        = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] console.ssl_ciphers            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] console.ssl_minimum_version    = default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] consoleauth.token_ttl          = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.cafile                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.certfile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.collect_timing          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.connect_retries         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.connect_retry_delay     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.endpoint_override       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.insecure                = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.keyfile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.max_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.min_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.region_name             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.service_name            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.service_type            = accelerator {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.split_loggers           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.status_code_retries     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.timeout                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.valid_interfaces        = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] cyborg.version                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.backend               = sqlalchemy {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.connection            = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.connection_debug      = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.connection_parameters =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.connection_recycle_time = 3600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.connection_trace      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.db_inc_retry_interval = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.db_max_retries        = 20 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.db_max_retry_interval = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.db_retry_interval     = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.max_overflow          = 50 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.max_pool_size         = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.max_retries           = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.mysql_enable_ndb      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.mysql_wsrep_sync_wait = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.pool_timeout          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.retry_interval        = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.slave_connection      = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] database.sqlite_synchronous    = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.backend           = sqlalchemy {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.connection        = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.connection_debug  = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.connection_parameters =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.connection_recycle_time = 3600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.connection_trace  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.db_inc_retry_interval = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.db_max_retries    = 20 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.db_max_retry_interval = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.db_retry_interval = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.max_overflow      = 50 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.max_pool_size     = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.max_retries       = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.mysql_enable_ndb  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.mysql_sql_mode    = TRADITIONAL {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.pool_timeout      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.retry_interval    = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.slave_connection  = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] api_database.sqlite_synchronous = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] devices.enabled_mdev_types     = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ephemeral_storage_encryption.enabled = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.api_servers             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.cafile                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.certfile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.collect_timing          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.connect_retries         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.connect_retry_delay     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.debug                   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.default_trusted_certificate_ids = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.enable_certificate_validation = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.enable_rbd_download     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.endpoint_override       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.insecure                = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.keyfile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.max_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.min_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.num_retries             = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.rbd_ceph_conf           =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.rbd_connect_timeout     = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.rbd_pool                =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.rbd_user                =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.region_name             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.service_name            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.service_type            = image {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.split_loggers           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.status_code_retries     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.timeout                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.valid_interfaces        = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.verify_glance_signatures = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] glance.version                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] guestfs.debug                  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.config_drive_cdrom      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.config_drive_inject_password = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.dynamic_memory_ratio    = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.enable_instance_metrics_collection = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.enable_remotefx         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.instances_path_share    =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.iscsi_initiator_list    = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.limit_cpu_features      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.power_state_check_timeframe = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.power_state_event_polling_interval = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.qemu_img_cmd            = qemu-img.exe {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.use_multipath_io        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.volume_attach_retry_count = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.volume_attach_retry_interval = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.vswitch_name            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] mks.enabled                    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] mks.mksproxy_base_url          = http://127.0.0.1:6090/ {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.manager_interval   = 2400 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.precache_concurrency = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.remove_unused_base_images = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] image_cache.subdirectory_name  = _base {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.api_max_retries         = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.api_retry_interval      = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.auth_section            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.auth_type               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.cafile                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.certfile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.collect_timing          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.connect_retries         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.connect_retry_delay     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.endpoint_override       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.insecure                = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.keyfile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.max_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.min_version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.partition_key           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.peer_list               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.region_name             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.serial_console_state_timeout = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.service_name            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.service_type            = baremetal {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.split_loggers           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.status_code_retries     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.timeout                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.valid_interfaces        = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ironic.version                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] key_manager.backend            = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] key_manager.fixed_key          = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.barbican_api_version  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.barbican_endpoint     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.barbican_endpoint_type = public {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.barbican_region_name  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.cafile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.certfile              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.collect_timing        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.insecure              = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.keyfile               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.number_of_retries     = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.retry_delay           = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.send_service_user_token = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.split_loggers         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.timeout               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.verify_ssl            = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican.verify_ssl_path       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.auth_section = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.auth_type = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.cafile   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.certfile = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.collect_timing = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.insecure = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.keyfile  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.split_loggers = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] barbican_service_user.timeout  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.approle_role_id          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.approle_secret_id        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.cafile                   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.certfile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.collect_timing           = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.insecure                 = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.keyfile                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.kv_mountpoint            = secret {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.kv_version               = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.namespace                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.root_token_id            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.split_loggers            = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.ssl_ca_crt_file          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.timeout                  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.use_ssl                  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.cafile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.certfile              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.collect_timing        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.connect_retries       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.connect_retry_delay   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.endpoint_override     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.insecure              = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.keyfile               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.max_version           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.min_version           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.region_name           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.service_name          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.service_type          = identity {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.split_loggers         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.status_code_retries   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.timeout               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.valid_interfaces      = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] keystone.version               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.connection_uri         =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_mode               = custom {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_model_extra_flags  = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: WARNING oslo_config.cfg [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt".
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_models             = ['Nehalem'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_power_governor_high = performance {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_power_governor_low = powersave {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_power_management   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.device_detach_attempts = 8 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.device_detach_timeout  = 20 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.disk_cachemodes        = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.disk_prefix            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.enabled_perf_events    = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.file_backed_memory     = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.gid_maps               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.hw_disk_discard        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.hw_machine_type        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_rbd_ceph_conf   =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_rbd_glance_store_name =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_rbd_pool        = rbd {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_type            = default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.images_volume_group    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.inject_key             = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.inject_partition       = -2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.inject_password        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.iscsi_iface            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.iser_use_multipath     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_bandwidth = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_downtime = 500 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_inbound_addr = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_permit_post_copy = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_scheme  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_timeout_action = abort {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_tunnelled = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: WARNING oslo_config.cfg [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal (
Apr 14 02:53:44 user nova-compute[71323]: live_migration_uri is deprecated for removal in favor of two other options that
Apr 14 02:53:44 user nova-compute[71323]: allow to change live migration scheme and target URI: ``live_migration_scheme``
Apr 14 02:53:44 user nova-compute[71323]: and ``live_migration_inbound_addr`` respectively.
Apr 14 02:53:44 user nova-compute[71323]: ).  Its value may be silently ignored in the future.
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_uri     = qemu+ssh://stack@%s/system {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.live_migration_with_native_tls = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.max_queues             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.nfs_mount_options      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.nfs_mount_point_base   = /opt/stack/data/nova/mnt {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_iser_scan_tries    = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_memory_encrypted_guests = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_pcie_ports         = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.num_volume_scan_tries  = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.pmem_namespaces        = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.quobyte_client_cfg     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rbd_connect_timeout    = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rbd_secret_uuid        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rbd_user               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.remote_filesystem_transport = ssh {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rescue_image_id        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rescue_kernel_id       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rescue_ramdisk_id      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rng_dev_path           = /dev/urandom {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.rx_queue_size          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.smbfs_mount_options    =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.snapshot_compression   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.snapshot_image_format  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.snapshots_directory    = /opt/stack/data/nova/instances/snapshots {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.sparse_logical_volumes = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.swtpm_enabled          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.swtpm_group            = tss {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.swtpm_user             = tss {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.sysinfo_serial         = unique {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.tx_queue_size          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.uid_maps               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.use_virtio_for_bridges = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.virt_type              = kvm {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.volume_clear           = zero {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.volume_clear_size      = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.volume_use_multipath   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_cache_path   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_log_path     = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_mount_group  = qemu {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_mount_opts   = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_mount_perms  = 0770 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.vzstorage_mount_user   = stack {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.auth_section           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.auth_type              = password {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.cafile                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.certfile               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.collect_timing         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.connect_retries        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.connect_retry_delay    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.default_floating_pool  = public {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.endpoint_override      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.extension_sync_interval = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.http_retries           = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.insecure               = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.keyfile                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.max_version            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.min_version            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.ovs_bridge             = br-int {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.physnets               = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.region_name            = RegionOne {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.service_metadata_proxy = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.service_name           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.service_type           = network {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.split_loggers          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.status_code_retries    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.timeout                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.valid_interfaces       = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] neutron.version                = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] notifications.bdms_in_notifications = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] notifications.default_level    = INFO {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] notifications.notification_format = unversioned {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] notifications.notify_on_state_change = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] pci.alias                      = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] pci.device_spec                = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] pci.report_in_placement        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.auth_section         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.auth_type            = password {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.auth_url             = http://10.0.0.210/identity {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.cafile               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.certfile             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.collect_timing       = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.connect_retries      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.connect_retry_delay  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.default_domain_id    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.default_domain_name  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.domain_id            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.domain_name          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.endpoint_override    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.insecure             = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.keyfile              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.max_version          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.min_version          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.password             = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.project_domain_id    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.project_domain_name  = Default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.project_id           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.project_name         = service {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.region_name          = RegionOne {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.service_name         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.service_type         = placement {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.split_loggers        = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.status_code_retries  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.system_scope         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.timeout              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.trust_id             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.user_domain_id       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.user_domain_name     = Default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.user_id              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.username             = placement {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.valid_interfaces     = ['internal', 'public'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] placement.version              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.cores                    = 20 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.count_usage_from_placement = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.driver                   = nova.quota.DbQuotaDriver {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.injected_file_content_bytes = 10240 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.injected_file_path_length = 255 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.injected_files           = 5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.instances                = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.key_pairs                = 100 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.metadata_items           = 128 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.ram                      = 51200 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.recheck_quota            = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.server_group_members     = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] quota.server_groups            = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rdp.enabled                    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] rdp.html5_proxy_base_url       = http://127.0.0.1:6083/ {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.image_metadata_prefilter = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.max_attempts         = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.max_placement_results = 1000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.query_placement_for_availability_zone = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.query_placement_for_image_type_support = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] scheduler.workers              = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.host_subset_size = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.isolated_hosts = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.isolated_images = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.pci_in_placement = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.track_instance_changes = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metrics.required               = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metrics.weight_multiplier      = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metrics.weight_of_unavailable  = -10000.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] metrics.weight_setting         = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.base_url        = ws://127.0.0.1:6083/ {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.enabled         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.port_range      = 10000:20000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] serial_console.serialproxy_port = 6083 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.auth_section      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.auth_type         = password {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.cafile            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.certfile          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.collect_timing    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.insecure          = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.keyfile           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.send_service_user_token = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.split_loggers     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] service_user.timeout           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.agent_enabled            = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.enabled                  = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.html5proxy_base_url      = http://10.0.0.210:6081/spice_auto.html {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.html5proxy_host          = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.html5proxy_port          = 6082 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.image_compression        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.jpeg_compression         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.playback_compression     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.server_listen            = 127.0.0.1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.streaming_mode           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] spice.zlib_compression         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] upgrade_levels.baseapi         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] upgrade_levels.cert            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] upgrade_levels.compute         = auto {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] upgrade_levels.conductor       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] upgrade_levels.scheduler       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.auth_section = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.auth_type = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.cafile = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.certfile = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.insecure = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.keyfile = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vendordata_dynamic_auth.timeout = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.api_retry_count         = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.ca_file                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.cache_prefix            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.cluster_name            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.connection_pool_size    = 10 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.console_delay_seconds   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.datastore_regex         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.host_ip                 = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.host_password           = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.host_port               = 443 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.host_username           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.insecure                = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.integration_bridge      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.maximum_objects         = 100 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.pbm_default_policy      = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.pbm_enabled             = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.pbm_wsdl_location       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.serial_log_dir          = /opt/vmware/vspc {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.serial_port_proxy_uri   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.serial_port_service_uri = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.task_poll_interval      = 0.5 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.use_linked_clone        = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.vnc_keymap              = en-us {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.vnc_port                = 5900 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vmware.vnc_port_total          = 10000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.auth_schemes               = ['none'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.enabled                    = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.novncproxy_base_url        = http://10.0.0.210:6080/vnc_lite.html {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.novncproxy_host            = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.novncproxy_port            = 6080 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.server_listen              = 0.0.0.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.server_proxyclient_address = 10.0.0.210 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.vencrypt_ca_certs          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.vencrypt_client_cert       = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vnc.vencrypt_client_key        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.disable_group_policy_check_upcall = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.disable_rootwrap   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.enable_numa_live_migration = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.libvirt_disable_apic = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.api_paste_config          = /etc/nova/api-paste.ini {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.client_socket_timeout     = 900 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.default_pool_size         = 1000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.keep_alive                = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.max_header_line           = 16384 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.secure_proxy_ssl_header   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.ssl_ca_file               = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.ssl_cert_file             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.ssl_key_file              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.tcp_keepidle              = 600 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] wsgi.wsgi_log_format           = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] zvm.ca_file                    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] zvm.cloud_connector_url        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] zvm.image_tmp_path             = /opt/stack/data/nova/images {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] zvm.reachable_timeout          = 300 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.enforce_new_defaults = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.enforce_scope      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.policy_default_rule = default {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.policy_file        = policy.yaml {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.connection_string     = messaging:// {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.enabled               = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.es_doc_type           = notification {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.es_scroll_size        = 10000 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.es_scroll_time        = 2m {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.filter_error_trace    = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.sentinel_service_name = mymaster {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.socket_timeout        = 0.1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] profiler.trace_sqlalchemy      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] remote_debug.host              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] remote_debug.port              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_notifications.retry = -1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_messaging_notifications.transport_url = **** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.auth_section        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.auth_type           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.cafile              = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.certfile            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.collect_timing      = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.connect_retries     = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.connect_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.endpoint_id         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.endpoint_override   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.insecure            = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.keyfile             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.max_version         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.min_version         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.region_name         = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.service_name        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.service_type        = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.split_loggers       = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.status_code_retries = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.status_code_retry_delay = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.timeout             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.valid_interfaces    = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_limit.version             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_reports.file_event_handler = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] oslo_reports.log_dir           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.capabilities = [12] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.group = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.helper_command = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.thread_pool_size = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_linux_bridge_privileged.user = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.capabilities = [12, 1] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.group  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.helper_command = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.thread_pool_size = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] vif_plug_ovs_privileged.user   = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.flat_interface = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.forward_bridge_interface = ['all'] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.iptables_bottom_regex =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.iptables_drop_action = DROP {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.iptables_top_regex =  {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.network_device_mtu = 1500 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.use_ipv6   = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_linux_bridge.vlan_interface = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.isolate_vif         = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.network_device_mtu  = 1500 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.ovs_vsctl_timeout   = 120 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.ovsdb_connection    = tcp:127.0.0.1:6640 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.ovsdb_interface     = native {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_vif_ovs.per_port_bridge     = False {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] os_brick.lock_path             = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.capabilities   = [21] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.group          = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.helper_command = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.logger_name    = os_brick.privileged {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.thread_pool_size = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] privsep_osbrick.user           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.capabilities    = [0, 1, 2, 3, 12, 21] {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.group           = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.helper_command  = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.logger_name     = oslo_privsep.daemon {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.thread_pool_size = 12 {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] nova_sys_admin.user            = None {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG oslo_service.service [None req-840f3116-b308-4309-b3b1-f0a93adb7d76 None None] ******************************************************************************** {{(pid=71323) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}}
Apr 14 02:53:44 user nova-compute[71323]: INFO nova.service [-] Starting compute node (version 0.0.0)
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Starting native event thread {{(pid=71323) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:492}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Starting green dispatch thread {{(pid=71323) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:498}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Starting connection event dispatch thread {{(pid=71323) initialize /opt/stack/nova/nova/virt/libvirt/host.py:620}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Connecting to libvirt: qemu:///system {{(pid=71323) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:503}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Registering for lifecycle events <nova.virt.libvirt.host.Host object at 0x7f2b18cbe650> {{(pid=71323) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:509}}
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7f2b18cbe650> {{(pid=71323) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:530}}
Apr 14 02:53:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Connection event '1' reason 'None'
Apr 14 02:53:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Cannot update service status on host "user" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 14 02:53:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.mount [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Initialising _HostMountState generation 0 {{(pid=71323) host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:130}}
Apr 14 02:53:51 user nova-compute[71323]: INFO nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host capabilities <capabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <host>
Apr 14 02:53:51 user nova-compute[71323]:     <uuid>e20c3142-5af9-7467-ecd8-70b2e4a210d6</uuid>
Apr 14 02:53:51 user nova-compute[71323]:     <cpu>
Apr 14 02:53:51 user nova-compute[71323]:       <arch>x86_64</arch>
Apr 14 02:53:51 user nova-compute[71323]:       <model>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <microcode version='1070'/>
Apr 14 02:53:51 user nova-compute[71323]:       <counter name='tsc' frequency='2094952000' scaling='no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <topology sockets='6' dies='1' cores='1' threads='1'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='osxsave'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='invtsc'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='rsba'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature name='skip-l1dfl-vmentry'/>
Apr 14 02:53:51 user nova-compute[71323]:       <pages unit='KiB' size='4'/>
Apr 14 02:53:51 user nova-compute[71323]:       <pages unit='KiB' size='2048'/>
Apr 14 02:53:51 user nova-compute[71323]:     </cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <power_management>
Apr 14 02:53:51 user nova-compute[71323]:       <suspend_mem/>
Apr 14 02:53:51 user nova-compute[71323]:       <suspend_disk/>
Apr 14 02:53:51 user nova-compute[71323]:       <suspend_hybrid/>
Apr 14 02:53:51 user nova-compute[71323]:     </power_management>
Apr 14 02:53:51 user nova-compute[71323]:     <iommu support='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <migration_features>
Apr 14 02:53:51 user nova-compute[71323]:       <live/>
Apr 14 02:53:51 user nova-compute[71323]:       <uri_transports>
Apr 14 02:53:51 user nova-compute[71323]:         <uri_transport>tcp</uri_transport>
Apr 14 02:53:51 user nova-compute[71323]:         <uri_transport>rdma</uri_transport>
Apr 14 02:53:51 user nova-compute[71323]:       </uri_transports>
Apr 14 02:53:51 user nova-compute[71323]:     </migration_features>
Apr 14 02:53:51 user nova-compute[71323]:     <topology>
Apr 14 02:53:51 user nova-compute[71323]:       <cells num='2'>
Apr 14 02:53:51 user nova-compute[71323]:         <cell id='0'>
Apr 14 02:53:51 user nova-compute[71323]:           <memory unit='KiB'>8189224</memory>
Apr 14 02:53:51 user nova-compute[71323]:           <pages unit='KiB' size='4'>2047306</pages>
Apr 14 02:53:51 user nova-compute[71323]:           <pages unit='KiB' size='2048'>0</pages>
Apr 14 02:53:51 user nova-compute[71323]:           <distances>
Apr 14 02:53:51 user nova-compute[71323]:             <sibling id='0' value='10'/>
Apr 14 02:53:51 user nova-compute[71323]:             <sibling id='1' value='20'/>
Apr 14 02:53:51 user nova-compute[71323]:           </distances>
Apr 14 02:53:51 user nova-compute[71323]:           <cpus num='6'>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='0' socket_id='0' die_id='0' core_id='0' siblings='0'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='1' socket_id='2' die_id='0' core_id='0' siblings='1'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='2' socket_id='4' die_id='0' core_id='0' siblings='2'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='3' socket_id='6' die_id='0' core_id='0' siblings='3'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='4' socket_id='8' die_id='0' core_id='0' siblings='4'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='5' socket_id='10' die_id='0' core_id='0' siblings='5'/>
Apr 14 02:53:51 user nova-compute[71323]:           </cpus>
Apr 14 02:53:51 user nova-compute[71323]:         </cell>
Apr 14 02:53:51 user nova-compute[71323]:         <cell id='1'>
Apr 14 02:53:51 user nova-compute[71323]:           <memory unit='KiB'>8218764</memory>
Apr 14 02:53:51 user nova-compute[71323]:           <pages unit='KiB' size='4'>2054691</pages>
Apr 14 02:53:51 user nova-compute[71323]:           <pages unit='KiB' size='2048'>0</pages>
Apr 14 02:53:51 user nova-compute[71323]:           <distances>
Apr 14 02:53:51 user nova-compute[71323]:             <sibling id='0' value='20'/>
Apr 14 02:53:51 user nova-compute[71323]:             <sibling id='1' value='10'/>
Apr 14 02:53:51 user nova-compute[71323]:           </distances>
Apr 14 02:53:51 user nova-compute[71323]:           <cpus num='6'>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='6' socket_id='12' die_id='0' core_id='0' siblings='6'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='7' socket_id='14' die_id='0' core_id='0' siblings='7'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='8' socket_id='16' die_id='0' core_id='0' siblings='8'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='9' socket_id='18' die_id='0' core_id='0' siblings='9'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='10' socket_id='20' die_id='0' core_id='0' siblings='10'/>
Apr 14 02:53:51 user nova-compute[71323]:             <cpu id='11' socket_id='22' die_id='0' core_id='0' siblings='11'/>
Apr 14 02:53:51 user nova-compute[71323]:           </cpus>
Apr 14 02:53:51 user nova-compute[71323]:         </cell>
Apr 14 02:53:51 user nova-compute[71323]:       </cells>
Apr 14 02:53:51 user nova-compute[71323]:     </topology>
Apr 14 02:53:51 user nova-compute[71323]:     <cache>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='2' level='3' type='both' size='15' unit='MiB' cpus='1'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='4' level='3' type='both' size='15' unit='MiB' cpus='2'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='6' level='3' type='both' size='15' unit='MiB' cpus='3'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='8' level='3' type='both' size='15' unit='MiB' cpus='4'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='10' level='3' type='both' size='15' unit='MiB' cpus='5'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='12' level='3' type='both' size='15' unit='MiB' cpus='6'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='14' level='3' type='both' size='15' unit='MiB' cpus='7'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='16' level='3' type='both' size='15' unit='MiB' cpus='8'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='18' level='3' type='both' size='15' unit='MiB' cpus='9'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='20' level='3' type='both' size='15' unit='MiB' cpus='10'/>
Apr 14 02:53:51 user nova-compute[71323]:       <bank id='22' level='3' type='both' size='15' unit='MiB' cpus='11'/>
Apr 14 02:53:51 user nova-compute[71323]:     </cache>
Apr 14 02:53:51 user nova-compute[71323]:     <secmodel>
Apr 14 02:53:51 user nova-compute[71323]:       <model>apparmor</model>
Apr 14 02:53:51 user nova-compute[71323]:       <doi>0</doi>
Apr 14 02:53:51 user nova-compute[71323]:     </secmodel>
Apr 14 02:53:51 user nova-compute[71323]:     <secmodel>
Apr 14 02:53:51 user nova-compute[71323]:       <model>dac</model>
Apr 14 02:53:51 user nova-compute[71323]:       <doi>0</doi>
Apr 14 02:53:51 user nova-compute[71323]:       <baselabel type='kvm'>+64055:+108</baselabel>
Apr 14 02:53:51 user nova-compute[71323]:       <baselabel type='qemu'>+64055:+108</baselabel>
Apr 14 02:53:51 user nova-compute[71323]:     </secmodel>
Apr 14 02:53:51 user nova-compute[71323]:   </host>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='alpha'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-alpha</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>clipper</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='armv6l'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>spitz</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>nuri</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>kzm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>z2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>tosa</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n810</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>terrier</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>midway</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>microbit</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>verdex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>highbank</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>akita</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>connex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>collie</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='armv7l'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>spitz</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>nuri</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>kzm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>z2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>tosa</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n810</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>terrier</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>midway</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>microbit</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>verdex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>highbank</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>akita</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>connex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>collie</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='aarch64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-aarch64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>integratorcp</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>ast2600-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>borzoi</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>spitz</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>nuri</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>romulus-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>npcm750-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>rainier-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps3-an547</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-b1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatileab</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>kzm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>musca-a</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>smdkc210</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>imx25-pdk</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>orangepi-pc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>z2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='6'>xlnx-zcu102</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>tosa</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps2-an521</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>sabrelite</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an511</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>canon-a1100</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-eb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>emcraft-sf2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>realview-pb-a8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>sbsa-ref</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi1ap</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>palmetto-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sx1-v1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n810</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g220a-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>n800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>tacoma-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>quanta-gsj</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>versatilepb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>terrier</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mainstone</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>mps3-an524</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>kudo-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>midway</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>musicpal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s811evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>lm3s6965evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>microbit</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an505</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an385</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>raspi3ap</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cubieboard</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>verdex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduino2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>xlnx-versal-virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mps2-an386</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>raspi3b</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>raspi2b</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>vexpress-a15</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2'>fuji-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='512'>virt-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>cheetah</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>virt-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ast2500-evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>highbank</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>akita</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>connex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>netduinoplus2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>collie</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>raspi0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <acpi default='on' toggle='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='cris'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-cris</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>axis-dev88</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='i686'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-i386</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>x-remote</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>isapc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>microvm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='kvm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <pae/>
Apr 14 02:53:51 user nova-compute[71323]:       <nonpae/>
Apr 14 02:53:51 user nova-compute[71323]:       <acpi default='on' toggle='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:       <apic default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='m68k'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-m68k</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mcf5208evb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>an5206</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virt-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>q800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virt-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='virt-6.2' maxCpus='1'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>next-cube</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virt-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='microblaze'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-microblaze</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='microblazeel'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-microblazeel</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>petalogix-ml605</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='mips'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-mips</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>malta</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='mipsel'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-mipsel</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>malta</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='mips64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-mips64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>malta</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>pica61</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>magnum</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='mips64el'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-mips64el</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>malta</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>loongson3-virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mipssim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>pica61</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>magnum</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='16'>boston</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>fuloong2e</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='ppc'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-ppc</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mac99</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>40p</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>taihu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='ppc64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-ppc64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>taihu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>40p</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mac99</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='ppc64le'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-ppc64le</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>taihu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='15'>mpc8544ds</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ppce500</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>40p</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>pegasos2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>bamboo</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>g3beige</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>virtex-ml507</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>mac99</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sam460ex</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>ref405ep</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2048'>powernv8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='riscv32'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-riscv32</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='8'>spike</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>opentitan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='5'>sifive_u</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sifive_e</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='8'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='riscv64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-riscv64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='8'>spike</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='5'>microchip-icicle-kit</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='5'>sifive_u</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>shakti_c</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sifive_e</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='8'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='s390x'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-s390x</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='s390-ccw-virtio-jammy' maxCpus='248'>s390-ccw-virtio</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='sh4'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-sh4</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>shix</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>r2d</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='sh4eb'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-sh4eb</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>shix</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>r2d</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='sparc'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-sparc</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>SS-5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>SS-20</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>LX</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>SPARCClassic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>leon3_generic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>SPARCbook</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>SS-4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>SS-600MP</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>SS-10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>Voyager</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='sparc64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-sparc64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sun4u</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>niagara</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>sun4v</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='x86_64'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>64</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>x-remote</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='1'>isapc</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-focal</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>microvm</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='kvm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <acpi default='on' toggle='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:       <apic default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='xtensa'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-xtensa</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>sim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>kc705</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ml605</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx200</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx60</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]:   <guest>
Apr 14 02:53:51 user nova-compute[71323]:     <os_type>hvm</os_type>
Apr 14 02:53:51 user nova-compute[71323]:     <arch name='xtensaeb'>
Apr 14 02:53:51 user nova-compute[71323]:       <wordsize>32</wordsize>
Apr 14 02:53:51 user nova-compute[71323]:       <emulator>/usr/bin/qemu-system-xtensaeb</emulator>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='4'>sim</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>kc705</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ml605</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>ml605-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>virt</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx60-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx200</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx200-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>lx60</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <machine maxCpus='32'>kc705-nommu</machine>
Apr 14 02:53:51 user nova-compute[71323]:       <domain type='qemu'/>
Apr 14 02:53:51 user nova-compute[71323]:     </arch>
Apr 14 02:53:51 user nova-compute[71323]:     <features>
Apr 14 02:53:51 user nova-compute[71323]:       <cpuselection/>
Apr 14 02:53:51 user nova-compute[71323]:       <deviceboot/>
Apr 14 02:53:51 user nova-compute[71323]:       <disksnapshot default='on' toggle='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     </features>
Apr 14 02:53:51 user nova-compute[71323]:   </guest>
Apr 14 02:53:51 user nova-compute[71323]: </capabilities>
Apr 14 02:53:51 user nova-compute[71323]: 
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for alpha via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch alpha / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for armv6l via machine types: {'virt', None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for armv7l via machine types: {'virt'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch armv7l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for aarch64 via machine types: {'virt'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch aarch64 / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for cris via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch cris / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for i686 via machine types: {'q35', 'ubuntu', 'ubuntu-q35', 'pc'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=q35:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-q35-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>i686</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='288'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'/>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='lm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-i440fx-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>i686</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='255'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'/>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='lm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ide</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu-q35:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-q35-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>i686</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='288'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'/>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='lm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=pc:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-i386</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-i440fx-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>i686</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='255'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'/>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='lm'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ide</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for m68k via machine types: {'virt', None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for microblaze via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch microblaze / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for microblazeel via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch microblazeel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for mips via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch mips / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for mipsel via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch mipsel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for mips64 via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch mips64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for mips64el via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch mips64el / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for ppc via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for ppc64 via machine types: {'powernv', 'pseries', None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for ppc64le via machine types: {'powernv', 'pseries'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for riscv32 via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch riscv32 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for riscv64 via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch riscv64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for s390x via machine types: {'s390-ccw-virtio'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch s390x / virt_type kvm / machine_type s390-ccw-virtio: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for sh4 via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch sh4 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for sh4eb via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch sh4eb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for sparc via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch sparc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for sparc64 via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch sparc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for x86_64 via machine types: {'q35', 'ubuntu', 'ubuntu-q35', 'pc'} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=q35:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-q35-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>x86_64</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='288'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>efi</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='invtsc'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='rsba'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-i440fx-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>x86_64</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='255'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>efi</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='invtsc'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='rsba'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ide</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu-q35:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-q35-jammy</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>x86_64</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='288'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>efi</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='invtsc'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='rsba'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=pc:
Apr 14 02:53:51 user nova-compute[71323]: <domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:   <path>/usr/bin/qemu-system-x86_64</path>
Apr 14 02:53:51 user nova-compute[71323]:   <domain>kvm</domain>
Apr 14 02:53:51 user nova-compute[71323]:   <machine>pc-i440fx-6.2</machine>
Apr 14 02:53:51 user nova-compute[71323]:   <arch>x86_64</arch>
Apr 14 02:53:51 user nova-compute[71323]:   <vcpu max='255'/>
Apr 14 02:53:51 user nova-compute[71323]:   <iothreads supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:   <os supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='firmware'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>efi</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:     <loader supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>rom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pflash</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='readonly'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>yes</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='secure'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>no</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </loader>
Apr 14 02:53:51 user nova-compute[71323]:   </os>
Apr 14 02:53:51 user nova-compute[71323]:   <cpu>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-passthrough' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='hostPassthroughMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='maximum' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='maximumMigratable'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>on</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>off</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='host-model' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <vendor>Intel</vendor>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ss'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='vmx'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pcid'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='hypervisor'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arat'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='tsc_adjust'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='umip'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='md-clear'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='arch-capabilities'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='invtsc'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibpb'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='ibrs'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-stibp'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='amd-ssbd'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='rsba'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='require' name='pschange-mc-no'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='erms'/>
Apr 14 02:53:51 user nova-compute[71323]:       <feature policy='disable' name='xsaveopt'/>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:     <mode name='custom' supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>qemu64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>qemu32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>phenom</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>pentium</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>n270</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm64</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>kvm32</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>coreduo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>core2duo</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>athlon</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Westmere</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Snowridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Skylake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>SandyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Penryn</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G5</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G4</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G3</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Opteron_G2</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Opteron_G1</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>IvyBridge</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Icelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Haswell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Rome</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-Milan</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC-IBPB</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>EPYC</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Dhyana</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cooperlake</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>Conroe</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Cascadelake-Server</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-noTSX</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell-IBRS</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='no'>Broadwell</model>
Apr 14 02:53:51 user nova-compute[71323]:       <model usable='yes'>486</model>
Apr 14 02:53:51 user nova-compute[71323]:     </mode>
Apr 14 02:53:51 user nova-compute[71323]:   </cpu>
Apr 14 02:53:51 user nova-compute[71323]:   <memoryBacking supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:     <enum name='sourceType'>
Apr 14 02:53:51 user nova-compute[71323]:       <value>file</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>anonymous</value>
Apr 14 02:53:51 user nova-compute[71323]:       <value>memfd</value>
Apr 14 02:53:51 user nova-compute[71323]:     </enum>
Apr 14 02:53:51 user nova-compute[71323]:   </memoryBacking>
Apr 14 02:53:51 user nova-compute[71323]:   <devices>
Apr 14 02:53:51 user nova-compute[71323]:     <disk supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='diskDevice'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>disk</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cdrom</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>floppy</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>lun</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='bus'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ide</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>fdc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sata</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </disk>
Apr 14 02:53:51 user nova-compute[71323]:     <graphics supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='type'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>sdl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vnc</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>spice</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egl-headless</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </graphics>
Apr 14 02:53:51 user nova-compute[71323]:     <video supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='modelType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>cirrus</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>vmvga</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>qxl</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>none</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>bochs</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>ramfb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </video>
Apr 14 02:53:51 user nova-compute[71323]:     <hostdev supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='mode'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>subsystem</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='startupPolicy'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>default</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>mandatory</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>requisite</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>optional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='subsysType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>usb</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>pci</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>scsi</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='capsType'/>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='pciBackend'/>
Apr 14 02:53:51 user nova-compute[71323]:     </hostdev>
Apr 14 02:53:51 user nova-compute[71323]:     <rng supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtio-non-transitional</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>random</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>egd</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>builtin</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </rng>
Apr 14 02:53:51 user nova-compute[71323]:     <filesystem supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='driverType'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>path</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>handle</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>virtiofs</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </filesystem>
Apr 14 02:53:51 user nova-compute[71323]:     <tpm supported='yes'>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='model'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-tis</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>tpm-crb</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:       <enum name='backendModel'>
Apr 14 02:53:51 user nova-compute[71323]:         <value>passthrough</value>
Apr 14 02:53:51 user nova-compute[71323]:         <value>emulator</value>
Apr 14 02:53:51 user nova-compute[71323]:       </enum>
Apr 14 02:53:51 user nova-compute[71323]:     </tpm>
Apr 14 02:53:51 user nova-compute[71323]:   </devices>
Apr 14 02:53:51 user nova-compute[71323]:   <features>
Apr 14 02:53:51 user nova-compute[71323]:     <gic supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:     <vmcoreinfo supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <genid supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backingStoreInput supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <backup supported='yes'/>
Apr 14 02:53:51 user nova-compute[71323]:     <sev supported='no'/>
Apr 14 02:53:51 user nova-compute[71323]:   </features>
Apr 14 02:53:51 user nova-compute[71323]: </domainCapabilities>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1037}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for xtensa via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch xtensa / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Getting domain capabilities for xtensaeb via machine types: {None} {{(pid=71323) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:952}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Error from libvirt when retrieving domain capabilities for arch xtensaeb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host {{(pid=71323) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:991}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Checking secure boot support for host arch (x86_64) {{(pid=71323) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1750}}
Apr 14 02:53:51 user nova-compute[71323]: INFO nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Secure Boot support detected
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] cpu compare xml: <cpu match="exact">
Apr 14 02:53:51 user nova-compute[71323]:   <model>Nehalem</model>
Apr 14 02:53:51 user nova-compute[71323]: </cpu>
Apr 14 02:53:51 user nova-compute[71323]:  {{(pid=71323) _compare_cpu /opt/stack/nova/nova/virt/libvirt/driver.py:9996}}
Apr 14 02:53:51 user nova-compute[71323]: INFO nova.virt.node [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Generated node identity 8eb2236f-d4b5-4afe-871b-37575055ded0
Apr 14 02:53:51 user nova-compute[71323]: INFO nova.virt.node [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Wrote node identity 8eb2236f-d4b5-4afe-871b-37575055ded0 to /opt/stack/data/nova/compute_id
Apr 14 02:53:51 user nova-compute[71323]: WARNING nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Compute nodes ['8eb2236f-d4b5-4afe-871b-37575055ded0'] for host user were not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.
Apr 14 02:53:51 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
Apr 14 02:53:51 user nova-compute[71323]: WARNING nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] No compute node record found for host user. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Apr 14 02:53:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:53:51 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:53:52 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:53:52 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Hypervisor/Node resource view: name=user free_ram=10822MB free_disk=27.23177719116211GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:53:52 user nova-compute[71323]: WARNING nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] No compute node record for user:8eb2236f-d4b5-4afe-871b-37575055ded0: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host 8eb2236f-d4b5-4afe-871b-37575055ded0 could not be found.
Apr 14 02:53:52 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Compute node record created for user:user with uuid: 8eb2236f-d4b5-4afe-871b-37575055ded0
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:53:52 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [req-ccae0a59-ed3f-4ff6-bf51-28a902f7dd83] Created resource provider record via placement API for resource provider with UUID 8eb2236f-d4b5-4afe-871b-37575055ded0 and name user.
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] /sys/module/kvm_amd/parameters/sev does not exist {{(pid=71323) _kernel_supports_amd_sev /opt/stack/nova/nova/virt/libvirt/host.py:1766}}
Apr 14 02:53:52 user nova-compute[71323]: INFO nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] kernel doesn't support AMD SEV
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Libvirt baseline CPU <cpu>
Apr 14 02:53:52 user nova-compute[71323]:   <arch>x86_64</arch>
Apr 14 02:53:52 user nova-compute[71323]:   <model>Nehalem</model>
Apr 14 02:53:52 user nova-compute[71323]:   <vendor>Intel</vendor>
Apr 14 02:53:52 user nova-compute[71323]:   <topology sockets="6" cores="1" threads="1"/>
Apr 14 02:53:52 user nova-compute[71323]: </cpu>
Apr 14 02:53:52 user nova-compute[71323]:  {{(pid=71323) _get_guest_baseline_cpu_features /opt/stack/nova/nova/virt/libvirt/driver.py:12486}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Updated inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with generation 0 in Placement from set_inventory_for_provider using data: {'MEMORY_MB': {'total': 16023, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:957}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Updating resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 generation from 0 to 1 during operation: update_inventory {{(pid=71323) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Updating resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 generation from 1 to 2 during operation: update_traits {{(pid=71323) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.639s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.service [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Creating RPC server for service compute {{(pid=71323) start /opt/stack/nova/nova/service.py:182}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.service [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Join ServiceGroup membership for this service compute {{(pid=71323) start /opt/stack/nova/nova/service.py:199}}
Apr 14 02:53:52 user nova-compute[71323]: DEBUG nova.servicegroup.drivers.db [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] DB_Driver: join new ServiceGroup member user to the compute group, service = <Service: host=user, binary=nova-compute, manager_class_name=nova.compute.manager.ComputeManager> {{(pid=71323) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}}
Apr 14 02:53:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:53:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:54:43 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:54:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:54:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=10192MB free_disk=27.145572662353516GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:54:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.107s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:55:45 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:55:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:55:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=10156MB free_disk=27.19000244140625GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:55:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.131s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:56:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:56:45 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:56:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:56:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=10136MB free_disk=26.96768569946289GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:56:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.187s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:56:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:56:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:47 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:57:48 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:57:48 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=9413MB free_disk=26.989608764648438GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:57:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.219s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:57:50 user nova-compute[71323]: INFO nova.compute.claims [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Claim successful on node user
Apr 14 02:57:50 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.322s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:57:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:57:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:57:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Creating image(s)
Apr 14 02:57:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "/opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "/opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "/opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG nova.policy [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0dbce21e767a4a61b9213e38523d247e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f021342634dd4f16bf2720533070154c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.part --force-share --output=json" returned: 0 in 0.200s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG nova.virt.images [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] 98e6a247-250b-4bb5-a947-e7f93598d16e was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.part /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.part /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.converted" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d.converted --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.416s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:52 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpkwjlf95i/privsep.sock']
Apr 14 02:57:52 user sudo[80263]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context nova.privsep.sys_admin_pctxt --privsep_sock_path /tmp/tmpkwjlf95i/privsep.sock
Apr 14 02:57:52 user sudo[80263]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 02:57:54 user sudo[80263]: pam_unix(sudo:session): session closed for user root
Apr 14 02:57:54 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Spawned new privsep daemon via rootwrap
Apr 14 02:57:54 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 02:57:54 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 02:57:54 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
Apr 14 02:57:54 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80266
Apr 14 02:57:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Successfully created port: f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:57:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk 1073741824" returned: 0 in 0.064s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.210s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Checking if we can resize image /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Cannot resize image /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'migration_context' on Instance uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Ensure instance console log exists: /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:57:55 user nova-compute[71323]: INFO nova.compute.claims [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Claim successful on node user
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.350s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:57:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:57:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Creating image(s)
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:57:56 user nova-compute[71323]: INFO nova.compute.claims [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Claim successful on node user
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.policy [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd6d78d4991b5493e8893034e91ed407b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '130665008cf648cf87467ae2777a9170', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.173s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:56 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk 1073741824" returned: 0 in 0.102s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.277s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.172s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking if we can resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.543s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Cannot resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.objects.instance [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'migration_context' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:57:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Ensure instance console log exists: /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:57:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Creating image(s)
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.policy [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd5e8150f1502427fb3148f20354d212e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '558bfd7e1d9049d396381de148b0c4a5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.202s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Successfully updated port: f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquired lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.180s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk 1073741824" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.234s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Checking if we can resize image /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Cannot resize image /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.objects.instance [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Ensure instance console log exists: /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-changed-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Refreshing instance network info cache due to event network-changed-f52f4b73-b4c4-4bc7-a565-1239a13770f1. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] Acquiring lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Successfully created port: 1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:57:58 user nova-compute[71323]: INFO nova.compute.claims [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Claim successful on node user
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updating instance_info_cache with network_info: [{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Releasing lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Instance network_info: |[{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] Acquired lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Refreshing network info cache for port f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Start _get_guest_xml network_info=[{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:57:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:57:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:57:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-934222543',display_name='tempest-ServersNegativeTestJSON-server-934222543',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-934222543',id=1,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-jc0pkunl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:51Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=1fb18db3-28b0-4886-a680-4c534d31cb6b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'pci_devices' on Instance uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.352s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:57:59 user nova-compute[71323]:   <uuid>1fb18db3-28b0-4886-a680-4c534d31cb6b</uuid>
Apr 14 02:57:59 user nova-compute[71323]:   <name>instance-00000001</name>
Apr 14 02:57:59 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:57:59 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:57:59 user nova-compute[71323]:   <metadata>
Apr 14 02:57:59 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:57:59 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:name>tempest-ServersNegativeTestJSON-server-934222543</nova:name>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:57:58</nova:creationTime>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:57:59 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:57:59 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:owner>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:user uuid="0dbce21e767a4a61b9213e38523d247e">tempest-ServersNegativeTestJSON-675126189-project-member</nova:user>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:project uuid="f021342634dd4f16bf2720533070154c">tempest-ServersNegativeTestJSON-675126189</nova:project>
Apr 14 02:57:59 user nova-compute[71323]:       </nova:owner>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:57:59 user nova-compute[71323]:       <nova:ports>
Apr 14 02:57:59 user nova-compute[71323]:         <nova:port uuid="f52f4b73-b4c4-4bc7-a565-1239a13770f1">
Apr 14 02:57:59 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 02:57:59 user nova-compute[71323]:         </nova:port>
Apr 14 02:57:59 user nova-compute[71323]:       </nova:ports>
Apr 14 02:57:59 user nova-compute[71323]:     </nova:instance>
Apr 14 02:57:59 user nova-compute[71323]:   </metadata>
Apr 14 02:57:59 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:57:59 user nova-compute[71323]:     <system>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="serial">1fb18db3-28b0-4886-a680-4c534d31cb6b</entry>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="uuid">1fb18db3-28b0-4886-a680-4c534d31cb6b</entry>
Apr 14 02:57:59 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:57:59 user nova-compute[71323]:     </system>
Apr 14 02:57:59 user nova-compute[71323]:   </sysinfo>
Apr 14 02:57:59 user nova-compute[71323]:   <os>
Apr 14 02:57:59 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:57:59 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:57:59 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:57:59 user nova-compute[71323]:   </os>
Apr 14 02:57:59 user nova-compute[71323]:   <features>
Apr 14 02:57:59 user nova-compute[71323]:     <acpi/>
Apr 14 02:57:59 user nova-compute[71323]:     <apic/>
Apr 14 02:57:59 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:57:59 user nova-compute[71323]:   </features>
Apr 14 02:57:59 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:57:59 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:57:59 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:57:59 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:57:59 user nova-compute[71323]:   </clock>
Apr 14 02:57:59 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:57:59 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:57:59 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:57:59 user nova-compute[71323]:   </cpu>
Apr 14 02:57:59 user nova-compute[71323]:   <devices>
Apr 14 02:57:59 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:57:59 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:57:59 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk"/>
Apr 14 02:57:59 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:57:59 user nova-compute[71323]:     </disk>
Apr 14 02:57:59 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:57:59 user nova-compute[71323]:       <mac address="fa:16:3e:d9:39:a8"/>
Apr 14 02:57:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:57:59 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:57:59 user nova-compute[71323]:       <target dev="tapf52f4b73-b4"/>
Apr 14 02:57:59 user nova-compute[71323]:     </interface>
Apr 14 02:57:59 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:57:59 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/console.log" append="off"/>
Apr 14 02:57:59 user nova-compute[71323]:     </serial>
Apr 14 02:57:59 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:57:59 user nova-compute[71323]:     <video>
Apr 14 02:57:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:57:59 user nova-compute[71323]:     </video>
Apr 14 02:57:59 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:57:59 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:57:59 user nova-compute[71323]:     </rng>
Apr 14 02:57:59 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:57:59 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:57:59 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:57:59 user nova-compute[71323]:     </memballoon>
Apr 14 02:57:59 user nova-compute[71323]:   </devices>
Apr 14 02:57:59 user nova-compute[71323]: </domain>
Apr 14 02:57:59 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-934222543',display_name='tempest-ServersNegativeTestJSON-server-934222543',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-934222543',id=1,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-jc0pkunl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:51Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=1fb18db3-28b0-4886-a680-4c534d31cb6b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG os_vif [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Created schema index Interface.name {{(pid=71323) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Created schema index Port.name {{(pid=71323) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Created schema index Bridge.name {{(pid=71323) autocreate_indices /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] tcp:127.0.0.1:6640: entering CONNECTING {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [POLLOUT] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 02:57:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:57:59 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmpwajvcezj/privsep.sock']
Apr 14 02:57:59 user sudo[80330]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpwajvcezj/privsep.sock
Apr 14 02:57:59 user sudo[80330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:57:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Creating image(s)
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.207s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG nova.policy [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd5e8150f1502427fb3148f20354d212e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '558bfd7e1d9049d396381de148b0c4a5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk 1073741824" returned: 0 in 0.061s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.199s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:57:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.190s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Cannot resize image /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Ensure instance console log exists: /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:58:00 user nova-compute[71323]: INFO nova.compute.claims [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Claim successful on node user
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updated VIF entry in instance network info cache for port f52f4b73-b4c4-4bc7-a565-1239a13770f1. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updating instance_info_cache with network_info: [{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad1e1bcf-6612-4402-9112-7586b866f3ec req-82d34245-1f74-419b-a487-9cc6405fb571 service nova] Releasing lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.398s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Successfully updated port: 1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Successfully created port: b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:58:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating image(s)
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:01 user sudo[80330]: pam_unix(sudo:session): session closed for user root
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.policy [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7dea23743ba945c4a9d539b8e277c653', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4075885e26cb4adaad24d3ae6217fe35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:58:01 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Spawned new privsep daemon via rootwrap
Apr 14 02:58:01 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 02:58:01 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 02:58:01 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_NET_ADMIN/CAP_DAC_OVERRIDE|CAP_NET_ADMIN/none
Apr 14 02:58:01 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 80350
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.188s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-changed-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Refreshing instance network info cache due to event network-changed-1319e09d-3a9f-404b-862d-37b385badb8d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk 1073741824" returned: 0 in 0.070s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.223s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Successfully created port: 59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Cannot resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'migration_context' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf52f4b73-b4, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf52f4b73-b4, col_values=(('external_ids', {'iface-id': 'f52f4b73-b4c4-4bc7-a565-1239a13770f1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d9:39:a8', 'vm-uuid': '1fb18db3-28b0-4886-a680-4c534d31cb6b'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Ensure instance console log exists: /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:01 user nova-compute[71323]: INFO os_vif [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4')
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] No VIF found with MAC fa:16:3e:d9:39:a8, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Successfully created port: 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance network_info: |[{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Refreshing network info cache for port 1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start _get_guest_xml network_info=[{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:02 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-846820438',display_name='tempest-ServerRescueNegativeTestJSON-server-846820438',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-846820438',id=3,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-5u5lcg43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:57Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=5e57cef1-b6cd-4291-a4c5-60e9dd22aed6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.objects.instance [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:03 user nova-compute[71323]:   <uuid>5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</uuid>
Apr 14 02:58:03 user nova-compute[71323]:   <name>instance-00000003</name>
Apr 14 02:58:03 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:03 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:03 user nova-compute[71323]:   <metadata>
Apr 14 02:58:03 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:03 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-846820438</nova:name>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:02</nova:creationTime>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:03 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:03 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 02:58:03 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:03 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:03 user nova-compute[71323]:         <nova:port uuid="1319e09d-3a9f-404b-862d-37b385badb8d">
Apr 14 02:58:03 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 02:58:03 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:03 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:03 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:03 user nova-compute[71323]:   </metadata>
Apr 14 02:58:03 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:03 user nova-compute[71323]:     <system>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="serial">5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</entry>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="uuid">5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</entry>
Apr 14 02:58:03 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:03 user nova-compute[71323]:     </system>
Apr 14 02:58:03 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:03 user nova-compute[71323]:   <os>
Apr 14 02:58:03 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:03 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:03 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:03 user nova-compute[71323]:   </os>
Apr 14 02:58:03 user nova-compute[71323]:   <features>
Apr 14 02:58:03 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:03 user nova-compute[71323]:     <apic/>
Apr 14 02:58:03 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:03 user nova-compute[71323]:   </features>
Apr 14 02:58:03 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:03 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:03 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:03 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:03 user nova-compute[71323]:   </clock>
Apr 14 02:58:03 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:03 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:03 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:03 user nova-compute[71323]:   </cpu>
Apr 14 02:58:03 user nova-compute[71323]:   <devices>
Apr 14 02:58:03 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:03 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:03 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk"/>
Apr 14 02:58:03 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:03 user nova-compute[71323]:     </disk>
Apr 14 02:58:03 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:03 user nova-compute[71323]:       <mac address="fa:16:3e:f3:f8:fa"/>
Apr 14 02:58:03 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:03 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:03 user nova-compute[71323]:       <target dev="tap1319e09d-3a"/>
Apr 14 02:58:03 user nova-compute[71323]:     </interface>
Apr 14 02:58:03 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:03 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/console.log" append="off"/>
Apr 14 02:58:03 user nova-compute[71323]:     </serial>
Apr 14 02:58:03 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:03 user nova-compute[71323]:     <video>
Apr 14 02:58:03 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:03 user nova-compute[71323]:     </video>
Apr 14 02:58:03 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:03 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:03 user nova-compute[71323]:     </rng>
Apr 14 02:58:03 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:03 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:03 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:03 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:03 user nova-compute[71323]:   </devices>
Apr 14 02:58:03 user nova-compute[71323]: </domain>
Apr 14 02:58:03 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-846820438',display_name='tempest-ServerRescueNegativeTestJSON-server-846820438',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-846820438',id=3,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-5u5lcg43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:57Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=5e57cef1-b6cd-4291-a4c5-60e9dd22aed6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG os_vif [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1319e09d-3a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1319e09d-3a, col_values=(('external_ids', {'iface-id': '1319e09d-3a9f-404b-862d-37b385badb8d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f3:f8:fa', 'vm-uuid': '5e57cef1-b6cd-4291-a4c5-60e9dd22aed6'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: INFO os_vif [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a')
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:f3:f8:fa, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Successfully updated port: b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Successfully updated port: 59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-changed-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Refreshing instance network info cache due to event network-changed-59fef74f-8c76-4dd5-bc99-311eea495950. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] Acquiring lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-changed-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Refreshing instance network info cache due to event network-changed-b85a839f-973d-415e-94b6-cdb480a8b89d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:04 user nova-compute[71323]: WARNING nova.compute.manager [req-809c2fa7-4f44-49cd-aa3d-df8b1e8ef4b1 req-69813013-2297-4625-8bbd-66b7062660dd service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state building and task_state spawning.
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Successfully updated port: 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance network_info: |[{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] Acquired lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Refreshing network info cache for port 59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start _get_guest_xml network_info=[{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updated VIF entry in instance network info cache for port 1319e09d-3a9f-404b-862d-37b385badb8d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d8dd314e-6769-4375-b88d-8f9333e04319 req-b0852546-5337-458f-a96e-8a38e46112f3 service nova] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:04 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:04 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1086696024',display_name='tempest-ServerRescueNegativeTestJSON-server-1086696024',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1086696024',id=4,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-d1yclxnx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:59Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=ac032176-63bc-457c-ad53-5b4a854eec34,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.neutron [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:05 user nova-compute[71323]:   <uuid>ac032176-63bc-457c-ad53-5b4a854eec34</uuid>
Apr 14 02:58:05 user nova-compute[71323]:   <name>instance-00000004</name>
Apr 14 02:58:05 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:05 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:05 user nova-compute[71323]:   <metadata>
Apr 14 02:58:05 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:05 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1086696024</nova:name>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:04</nova:creationTime>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:05 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:port uuid="59fef74f-8c76-4dd5-bc99-311eea495950">
Apr 14 02:58:05 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 02:58:05 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:05 user nova-compute[71323]:   </metadata>
Apr 14 02:58:05 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:05 user nova-compute[71323]:     <system>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="serial">ac032176-63bc-457c-ad53-5b4a854eec34</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="uuid">ac032176-63bc-457c-ad53-5b4a854eec34</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:05 user nova-compute[71323]:     </system>
Apr 14 02:58:05 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:05 user nova-compute[71323]:   <os>
Apr 14 02:58:05 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:05 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:05 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:05 user nova-compute[71323]:   </os>
Apr 14 02:58:05 user nova-compute[71323]:   <features>
Apr 14 02:58:05 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:05 user nova-compute[71323]:     <apic/>
Apr 14 02:58:05 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:05 user nova-compute[71323]:   </features>
Apr 14 02:58:05 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:05 user nova-compute[71323]:   </clock>
Apr 14 02:58:05 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:05 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:05 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:05 user nova-compute[71323]:   </cpu>
Apr 14 02:58:05 user nova-compute[71323]:   <devices>
Apr 14 02:58:05 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:05 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:05 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </disk>
Apr 14 02:58:05 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:05 user nova-compute[71323]:       <mac address="fa:16:3e:b2:39:e3"/>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="tap59fef74f-8c"/>
Apr 14 02:58:05 user nova-compute[71323]:     </interface>
Apr 14 02:58:05 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:05 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/console.log" append="off"/>
Apr 14 02:58:05 user nova-compute[71323]:     </serial>
Apr 14 02:58:05 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:     <video>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </video>
Apr 14 02:58:05 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:05 user nova-compute[71323]:     </rng>
Apr 14 02:58:05 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:05 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:05 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:05 user nova-compute[71323]:   </devices>
Apr 14 02:58:05 user nova-compute[71323]: </domain>
Apr 14 02:58:05 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1086696024',display_name='tempest-ServerRescueNegativeTestJSON-server-1086696024',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1086696024',id=4,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-d1yclxnx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:59Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=ac032176-63bc-457c-ad53-5b4a854eec34,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG os_vif [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance network_info: |[{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.neutron [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Refreshing network info cache for port b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start _get_guest_xml network_info=[{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap59fef74f-8c, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap59fef74f-8c, col_values=(('external_ids', {'iface-id': '59fef74f-8c76-4dd5-bc99-311eea495950', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b2:39:e3', 'vm-uuid': 'ac032176-63bc-457c-ad53-5b4a854eec34'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: INFO os_vif [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c')
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:56Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'pci_devices' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:05 user nova-compute[71323]:   <uuid>bbb6cd58-ce7a-43ae-876d-51a888193f60</uuid>
Apr 14 02:58:05 user nova-compute[71323]:   <name>instance-00000002</name>
Apr 14 02:58:05 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:05 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:05 user nova-compute[71323]:   <metadata>
Apr 14 02:58:05 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:05 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeTestJSON-server-1854200462</nova:name>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:05</nova:creationTime>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:05 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:user uuid="d6d78d4991b5493e8893034e91ed407b">tempest-AttachVolumeTestJSON-975876676-project-member</nova:user>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:project uuid="130665008cf648cf87467ae2777a9170">tempest-AttachVolumeTestJSON-975876676</nova:project>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:port uuid="b85a839f-973d-415e-94b6-cdb480a8b89d">
Apr 14 02:58:05 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 02:58:05 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:05 user nova-compute[71323]:   </metadata>
Apr 14 02:58:05 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:05 user nova-compute[71323]:     <system>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="serial">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="uuid">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:05 user nova-compute[71323]:     </system>
Apr 14 02:58:05 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:05 user nova-compute[71323]:   <os>
Apr 14 02:58:05 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:05 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:05 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:05 user nova-compute[71323]:   </os>
Apr 14 02:58:05 user nova-compute[71323]:   <features>
Apr 14 02:58:05 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:05 user nova-compute[71323]:     <apic/>
Apr 14 02:58:05 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:05 user nova-compute[71323]:   </features>
Apr 14 02:58:05 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:05 user nova-compute[71323]:   </clock>
Apr 14 02:58:05 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:05 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:05 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:05 user nova-compute[71323]:   </cpu>
Apr 14 02:58:05 user nova-compute[71323]:   <devices>
Apr 14 02:58:05 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:05 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:05 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </disk>
Apr 14 02:58:05 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:05 user nova-compute[71323]:       <mac address="fa:16:3e:f7:89:a6"/>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="tapb85a839f-97"/>
Apr 14 02:58:05 user nova-compute[71323]:     </interface>
Apr 14 02:58:05 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:05 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/console.log" append="off"/>
Apr 14 02:58:05 user nova-compute[71323]:     </serial>
Apr 14 02:58:05 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:     <video>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </video>
Apr 14 02:58:05 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:05 user nova-compute[71323]:     </rng>
Apr 14 02:58:05 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:05 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:05 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:05 user nova-compute[71323]:   </devices>
Apr 14 02:58:05 user nova-compute[71323]: </domain>
Apr 14 02:58:05 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:57:56Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG os_vif [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb85a839f-97, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb85a839f-97, col_values=(('external_ids', {'iface-id': 'b85a839f-973d-415e-94b6-cdb480a8b89d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:89:a6', 'vm-uuid': 'bbb6cd58-ce7a-43ae-876d-51a888193f60'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: INFO os_vif [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:b2:39:e3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No VIF found with MAC fa:16:3e:f7:89:a6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance network_info: |[{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start _get_guest_xml network_info=[{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1395552505',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:05 user nova-compute[71323]:   <uuid>6bb082bc-0c74-4149-9bd0-ceb09d7764d4</uuid>
Apr 14 02:58:05 user nova-compute[71323]:   <name>instance-00000005</name>
Apr 14 02:58:05 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:05 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:05 user nova-compute[71323]:   <metadata>
Apr 14 02:58:05 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:05 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-1395552505</nova:name>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:05</nova:creationTime>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:05 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:05 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:         <nova:port uuid="04b5c1f1-8988-4a34-b97a-ad0cecd12b71">
Apr 14 02:58:05 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 02:58:05 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:05 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:05 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:05 user nova-compute[71323]:   </metadata>
Apr 14 02:58:05 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:05 user nova-compute[71323]:     <system>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="serial">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="uuid">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 02:58:05 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:05 user nova-compute[71323]:     </system>
Apr 14 02:58:05 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:05 user nova-compute[71323]:   <os>
Apr 14 02:58:05 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:05 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:05 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:05 user nova-compute[71323]:   </os>
Apr 14 02:58:05 user nova-compute[71323]:   <features>
Apr 14 02:58:05 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:05 user nova-compute[71323]:     <apic/>
Apr 14 02:58:05 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:05 user nova-compute[71323]:   </features>
Apr 14 02:58:05 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:05 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:05 user nova-compute[71323]:   </clock>
Apr 14 02:58:05 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:05 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:05 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:05 user nova-compute[71323]:   </cpu>
Apr 14 02:58:05 user nova-compute[71323]:   <devices>
Apr 14 02:58:05 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:05 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:05 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </disk>
Apr 14 02:58:05 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:05 user nova-compute[71323]:       <mac address="fa:16:3e:af:bb:dd"/>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:05 user nova-compute[71323]:       <target dev="tap04b5c1f1-89"/>
Apr 14 02:58:05 user nova-compute[71323]:     </interface>
Apr 14 02:58:05 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:05 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log" append="off"/>
Apr 14 02:58:05 user nova-compute[71323]:     </serial>
Apr 14 02:58:05 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:05 user nova-compute[71323]:     <video>
Apr 14 02:58:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:05 user nova-compute[71323]:     </video>
Apr 14 02:58:05 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:05 user nova-compute[71323]:     </rng>
Apr 14 02:58:05 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:05 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:05 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:05 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:05 user nova-compute[71323]:   </devices>
Apr 14 02:58:05 user nova-compute[71323]: </domain>
Apr 14 02:58:05 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1395552505',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG os_vif [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap04b5c1f1-89, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap04b5c1f1-89, col_values=(('external_ids', {'iface-id': '04b5c1f1-8988-4a34-b97a-ad0cecd12b71', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:af:bb:dd', 'vm-uuid': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:05 user nova-compute[71323]: INFO os_vif [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.neutron [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updated VIF entry in instance network info cache for port 59fef74f-8c76-4dd5-bc99-311eea495950. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.network.neutron [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08f77f53-d616-49da-a819-31579f20010d req-ed30c39a-f4a8-4cd9-9985-e915702ca276 service nova] Releasing lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:af:bb:dd, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-changed-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Refreshing instance network info cache due to event network-changed-04b5c1f1-8988-4a34-b97a-ad0cecd12b71. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.network.neutron [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Refreshing network info cache for port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441086.2984946, 1fb18db3-28b0-4886-a680-4c534d31cb6b => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] VM Resumed (Lifecycle Event)
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.claims [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Claim successful on node user
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Instance spawned successfully.
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Took 15.37 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441086.298615, 1fb18db3-28b0-4886-a680-4c534d31cb6b => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] VM Started (Lifecycle Event)
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:06 user nova-compute[71323]: INFO nova.compute.manager [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Took 16.25 seconds to build instance.
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-360cfb37-5a85-4da7-a5d9-a8b0ad63b6aa tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.539s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:06 user nova-compute[71323]: WARNING nova.compute.manager [req-0cf65505-9140-44d8-9203-9ab813b0e052 req-f678a086-f6b7-49bc-ad6b-dd3a5606b343 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state building and task_state spawning.
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 1.205s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.network.neutron [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updated VIF entry in instance network info cache for port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.network.neutron [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:07 user nova-compute[71323]: WARNING nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state building and task_state spawning.
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:07 user nova-compute[71323]: WARNING nova.compute.manager [req-0deb4683-e377-4019-8d3c-404aa1b4ecf3 req-f27a6fcf-205a-4fb2-b621-d1bcdbf31766 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state building and task_state spawning.
Apr 14 02:58:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.policy [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '70eb1e59ba1342688a8ce4e6ff440f58', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46cec85cd4364c8eaad6870b92249a56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.network.neutron [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updated VIF entry in instance network info cache for port b85a839f-973d-415e-94b6-cdb480a8b89d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.network.neutron [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e3a7dda-7a6b-4579-b3c9-ee896851fcfc req-d397c35a-59ef-4f1a-adfd-f4af067ea700 service nova] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:58:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Creating image(s)
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "/opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "/opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "/opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.163s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk 1073741824" returned: 0 in 0.095s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.253s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.179s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Checking if we can resize image /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Cannot resize image /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.objects.instance [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lazy-loading 'migration_context' on Instance uuid 82a9ec77-951b-4394-b555-00a49710acb5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Ensure instance console log exists: /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Successfully created port: 35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:09 user nova-compute[71323]: WARNING nova.compute.manager [req-9072a6db-5c79-4712-8ea9-f5fa4849ee2e req-c2a33de7-d589-46a6-b497-889b4dfe9c9b service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state building and task_state spawning.
Apr 14 02:58:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Successfully updated port: 35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquired lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-changed-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Refreshing instance network info cache due to event network-changed-35c25a8a-1450-4b7a-8a1f-11c047bb9d27. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] Acquiring lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Updating instance_info_cache with network_info: [{"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Releasing lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Instance network_info: |[{"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] Acquired lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Refreshing network info cache for port 35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Start _get_guest_xml network_info=[{"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-815548039',display_name='tempest-DeleteServersTestJSON-server-815548039',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-815548039',id=6,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46cec85cd4364c8eaad6870b92249a56',ramdisk_id='',reservation_id='r-is0fgfby',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-360109579',owner_user_name='tempest-DeleteServersTestJSON-360109579-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:07Z,user_data=None,user_id='70eb1e59ba1342688a8ce4e6ff440f58',uuid=82a9ec77-951b-4394-b555-00a49710acb5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converting VIF {"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.objects.instance [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lazy-loading 'pci_devices' on Instance uuid 82a9ec77-951b-4394-b555-00a49710acb5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:10 user nova-compute[71323]: WARNING nova.compute.manager [req-3b9bdb67-f86f-46bb-a108-017f91905506 req-3a24881b-3beb-41a1-a0d9-696ae4df8216 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state building and task_state spawning.
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:10 user nova-compute[71323]:   <uuid>82a9ec77-951b-4394-b555-00a49710acb5</uuid>
Apr 14 02:58:10 user nova-compute[71323]:   <name>instance-00000006</name>
Apr 14 02:58:10 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:10 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:10 user nova-compute[71323]:   <metadata>
Apr 14 02:58:10 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:10 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:name>tempest-DeleteServersTestJSON-server-815548039</nova:name>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:10</nova:creationTime>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:10 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:10 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:user uuid="70eb1e59ba1342688a8ce4e6ff440f58">tempest-DeleteServersTestJSON-360109579-project-member</nova:user>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:project uuid="46cec85cd4364c8eaad6870b92249a56">tempest-DeleteServersTestJSON-360109579</nova:project>
Apr 14 02:58:10 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:10 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:10 user nova-compute[71323]:         <nova:port uuid="35c25a8a-1450-4b7a-8a1f-11c047bb9d27">
Apr 14 02:58:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 02:58:10 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:10 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:10 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:10 user nova-compute[71323]:   </metadata>
Apr 14 02:58:10 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:10 user nova-compute[71323]:     <system>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="serial">82a9ec77-951b-4394-b555-00a49710acb5</entry>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="uuid">82a9ec77-951b-4394-b555-00a49710acb5</entry>
Apr 14 02:58:10 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:10 user nova-compute[71323]:     </system>
Apr 14 02:58:10 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:10 user nova-compute[71323]:   <os>
Apr 14 02:58:10 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:10 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:10 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:10 user nova-compute[71323]:   </os>
Apr 14 02:58:10 user nova-compute[71323]:   <features>
Apr 14 02:58:10 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:10 user nova-compute[71323]:     <apic/>
Apr 14 02:58:10 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:10 user nova-compute[71323]:   </features>
Apr 14 02:58:10 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:10 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:10 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:10 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:10 user nova-compute[71323]:   </clock>
Apr 14 02:58:10 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:10 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:10 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:10 user nova-compute[71323]:   </cpu>
Apr 14 02:58:10 user nova-compute[71323]:   <devices>
Apr 14 02:58:10 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:10 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:10 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk"/>
Apr 14 02:58:10 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:10 user nova-compute[71323]:     </disk>
Apr 14 02:58:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:10 user nova-compute[71323]:       <mac address="fa:16:3e:13:c8:bf"/>
Apr 14 02:58:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:10 user nova-compute[71323]:       <target dev="tap35c25a8a-14"/>
Apr 14 02:58:10 user nova-compute[71323]:     </interface>
Apr 14 02:58:10 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:10 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/console.log" append="off"/>
Apr 14 02:58:10 user nova-compute[71323]:     </serial>
Apr 14 02:58:10 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:10 user nova-compute[71323]:     <video>
Apr 14 02:58:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:10 user nova-compute[71323]:     </video>
Apr 14 02:58:10 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:10 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:10 user nova-compute[71323]:     </rng>
Apr 14 02:58:10 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:10 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:10 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:10 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:10 user nova-compute[71323]:   </devices>
Apr 14 02:58:10 user nova-compute[71323]: </domain>
Apr 14 02:58:10 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-815548039',display_name='tempest-DeleteServersTestJSON-server-815548039',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-815548039',id=6,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46cec85cd4364c8eaad6870b92249a56',ramdisk_id='',reservation_id='r-is0fgfby',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-360109579',owner_user_name='tempest-DeleteServersTestJSON-360109579-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:07Z,user_data=None,user_id='70eb1e59ba1342688a8ce4e6ff440f58',uuid=82a9ec77-951b-4394-b555-00a49710acb5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converting VIF {"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG os_vif [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap35c25a8a-14, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap35c25a8a-14, col_values=(('external_ids', {'iface-id': '35c25a8a-1450-4b7a-8a1f-11c047bb9d27', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:13:c8:bf', 'vm-uuid': '82a9ec77-951b-4394-b555-00a49710acb5'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:10 user nova-compute[71323]: INFO os_vif [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14')
Apr 14 02:58:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] No VIF found with MAC fa:16:3e:13:c8:bf, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:11 user nova-compute[71323]: WARNING nova.compute.manager [req-9c89dd68-d1c7-456d-a181-b23251d6761a req-16101964-8dde-4aa6-8be7-cfdacac00073 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state building and task_state spawning.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:12 user nova-compute[71323]: WARNING nova.compute.manager [req-6a6c96e3-fe91-4f8d-8a4c-26e9cccb3c97 req-481946ae-224e-485e-8978-66df5071776d service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state building and task_state spawning.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Updated VIF entry in instance network info cache for port 35c25a8a-1450-4b7a-8a1f-11c047bb9d27. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Updating instance_info_cache with network_info: [{"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36440a39-160e-4da3-964b-66ec49a43015 req-761c3f17-8afa-4bc4-9b3c-8387a8c623e8 service nova] Releasing lock "refresh_cache-82a9ec77-951b-4394-b555-00a49710acb5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441092.4428704, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Resumed (Lifecycle Event)
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance spawned successfully.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441092.4429815, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Started (Lifecycle Event)
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Took 15.14 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:12 user nova-compute[71323]: INFO nova.compute.manager [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Took 16.28 seconds to build instance.
Apr 14 02:58:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-75fddbaa-e056-4e56-9ce0-c76737f7e3f4 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.504s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:13 user nova-compute[71323]: WARNING nova.compute.manager [req-dc46105f-1e14-4a63-97d9-98ee53e77ab8 req-5b1927a1-4377-4b84-aac1-e83084d9a51a service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state building and task_state spawning.
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441094.0796452, ac032176-63bc-457c-ad53-5b4a854eec34 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] VM Resumed (Lifecycle Event)
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance spawned successfully.
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:14 user nova-compute[71323]: WARNING nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state building and task_state spawning.
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:14 user nova-compute[71323]: WARNING nova.compute.manager [req-c578d502-8a85-40bc-ac7a-0383ab03354d req-2c92bbc5-c54b-45e6-91ab-7794c92c52e1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state building and task_state spawning.
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441094.082081, ac032176-63bc-457c-ad53-5b4a854eec34 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] VM Started (Lifecycle Event)
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Took 14.87 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:14 user nova-compute[71323]: INFO nova.compute.manager [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Took 15.90 seconds to build instance.
Apr 14 02:58:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e2bb68ae-7cfd-4970-977d-df2358c4cb02 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.047s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441095.2031047, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Resumed (Lifecycle Event)
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance spawned successfully.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441095.2066488, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Started (Lifecycle Event)
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance spawned successfully.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441095.361828, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Resumed (Lifecycle Event)
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Took 18.91 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441095.3634157, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Started (Lifecycle Event)
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:15 user nova-compute[71323]: INFO nova.compute.manager [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Took 20.10 seconds to build instance.
Apr 14 02:58:16 user nova-compute[71323]: INFO nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Took 14.93 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-874dc327-6a8f-47eb-9312-04e595bf1f37 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 20.454s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:16 user nova-compute[71323]: WARNING nova.compute.manager [req-652d88eb-c716-4d87-876f-3d8854dc74c9 req-72d36c9a-c5b0-41e4-972e-f0a7dc53aaa3 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state building and task_state spawning.
Apr 14 02:58:16 user nova-compute[71323]: INFO nova.compute.manager [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Took 16.11 seconds to build instance.
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5bfca42b-717f-47c5-8479-f20eb4bcbe6f tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.314s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441096.8905773, 82a9ec77-951b-4394-b555-00a49710acb5 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] VM Resumed (Lifecycle Event)
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:16 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Instance spawned successfully.
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441096.8906815, 82a9ec77-951b-4394-b555-00a49710acb5 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] VM Started (Lifecycle Event)
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Took 9.71 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.claims [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Claim successful on node user
Apr 14 02:58:17 user nova-compute[71323]: INFO nova.compute.manager [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Took 11.67 seconds to build instance.
Apr 14 02:58:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-57c10d87-f58c-4906-a225-9a698f892272 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.905s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 1.272s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:58:18 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.policy [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f0d4916eb93c4b3ebbbd2af4b0cdacc1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94a1eb61159b43228db0d529a5be816d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:58:18 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Creating image(s)
Apr 14 02:58:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "/opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "/opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "/opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.010s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.259s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.277s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk 1073741824" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.421s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.241s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:58:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.236s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Cannot resize image /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG nova.objects.instance [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'migration_context' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Ensure instance console log exists: /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.006s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Successfully created port: 9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Successfully updated port: 9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquired lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-changed-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Refreshing instance network info cache due to event network-changed-9a41964d-ca86-41d7-8287-1af6e94403b4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] Acquiring lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating instance_info_cache with network_info: [{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Releasing lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Instance network_info: |[{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] Acquired lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Refreshing network info cache for port 9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Start _get_guest_xml network_info=[{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:22 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:22 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1530771040',display_name='tempest-VolumesAdminNegativeTest-server-1530771040',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1530771040',id=7,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLRjkf56mNaAsNKFgnUHc6YtgNOI11oCC45IpAljzRBHWdlhRZJtff4EnryoIR2UUdedBXj1eNEYfP5CFJHsIuaLL73yA24OQjZOf+7GA5PGlabqvgSGMvLYjqcMcQgnmA==',key_name='tempest-keypair-309889138',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-klkjfltg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'pci_devices' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:22 user nova-compute[71323]:   <uuid>3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5</uuid>
Apr 14 02:58:22 user nova-compute[71323]:   <name>instance-00000007</name>
Apr 14 02:58:22 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:22 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:22 user nova-compute[71323]:   <metadata>
Apr 14 02:58:22 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:22 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1530771040</nova:name>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:22</nova:creationTime>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:22 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:22 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:user uuid="f0d4916eb93c4b3ebbbd2af4b0cdacc1">tempest-VolumesAdminNegativeTest-1232189050-project-member</nova:user>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:project uuid="94a1eb61159b43228db0d529a5be816d">tempest-VolumesAdminNegativeTest-1232189050</nova:project>
Apr 14 02:58:22 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:58:22 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:22 user nova-compute[71323]:         <nova:port uuid="9a41964d-ca86-41d7-8287-1af6e94403b4">
Apr 14 02:58:22 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 02:58:22 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:22 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:22 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:22 user nova-compute[71323]:   </metadata>
Apr 14 02:58:22 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:22 user nova-compute[71323]:     <system>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="serial">3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5</entry>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="uuid">3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5</entry>
Apr 14 02:58:22 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:22 user nova-compute[71323]:     </system>
Apr 14 02:58:22 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:22 user nova-compute[71323]:   <os>
Apr 14 02:58:22 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:22 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:22 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:22 user nova-compute[71323]:   </os>
Apr 14 02:58:22 user nova-compute[71323]:   <features>
Apr 14 02:58:22 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:22 user nova-compute[71323]:     <apic/>
Apr 14 02:58:22 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:22 user nova-compute[71323]:   </features>
Apr 14 02:58:22 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:22 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:22 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:22 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:22 user nova-compute[71323]:   </clock>
Apr 14 02:58:22 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:22 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:22 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:22 user nova-compute[71323]:   </cpu>
Apr 14 02:58:22 user nova-compute[71323]:   <devices>
Apr 14 02:58:22 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:22 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:22 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk"/>
Apr 14 02:58:22 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:58:22 user nova-compute[71323]:     </disk>
Apr 14 02:58:22 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:22 user nova-compute[71323]:       <mac address="fa:16:3e:7a:2f:92"/>
Apr 14 02:58:22 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:22 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:22 user nova-compute[71323]:       <target dev="tap9a41964d-ca"/>
Apr 14 02:58:22 user nova-compute[71323]:     </interface>
Apr 14 02:58:22 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:22 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/console.log" append="off"/>
Apr 14 02:58:22 user nova-compute[71323]:     </serial>
Apr 14 02:58:22 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:22 user nova-compute[71323]:     <video>
Apr 14 02:58:22 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:22 user nova-compute[71323]:     </video>
Apr 14 02:58:22 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:22 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:22 user nova-compute[71323]:     </rng>
Apr 14 02:58:22 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:22 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:22 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:22 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:22 user nova-compute[71323]:   </devices>
Apr 14 02:58:22 user nova-compute[71323]: </domain>
Apr 14 02:58:22 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1530771040',display_name='tempest-VolumesAdminNegativeTest-server-1530771040',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1530771040',id=7,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLRjkf56mNaAsNKFgnUHc6YtgNOI11oCC45IpAljzRBHWdlhRZJtff4EnryoIR2UUdedBXj1eNEYfP5CFJHsIuaLL73yA24OQjZOf+7GA5PGlabqvgSGMvLYjqcMcQgnmA==',key_name='tempest-keypair-309889138',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-klkjfltg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG os_vif [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9a41964d-ca, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap9a41964d-ca, col_values=(('external_ids', {'iface-id': '9a41964d-ca86-41d7-8287-1af6e94403b4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7a:2f:92', 'vm-uuid': '3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:22 user nova-compute[71323]: INFO os_vif [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca')
Apr 14 02:58:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No VIF found with MAC fa:16:3e:7a:2f:92, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updated VIF entry in instance network info cache for port 9a41964d-ca86-41d7-8287-1af6e94403b4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:58:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating instance_info_cache with network_info: [{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1d244dbd-54c4-4a2f-b153-73abc6790d45 req-ccf8b811-8dbe-4b6a-bdba-e5b6e37b0285 service nova] Releasing lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] No waiting events found dispatching network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:25 user nova-compute[71323]: WARNING nova.compute.manager [req-a006a6f9-47d9-4dfb-ac31-f473111eb9c1 req-eec01220-d356-4aeb-b1a0-a465c5aa6923 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received unexpected event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 for instance with vm_state building and task_state spawning.
Apr 14 02:58:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:58:26 user nova-compute[71323]: INFO nova.compute.claims [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Claim successful on node user
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.658s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:58:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Ignoring supplied device name: /dev/sda. Libvirt can't honour user-supplied dev names
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.policy [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9eabf8254cde4d86a1862934ec9b5240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcbeb4e2fdc1420c98c264aac62340a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:58:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Creating image(s)
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] No waiting events found dispatching network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:27 user nova-compute[71323]: WARNING nova.compute.manager [req-5a6efb38-7723-4331-8978-c37eaff90083 req-4aa9156d-672d-48c1-9dc9-87b2e9c7c64a service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received unexpected event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 for instance with vm_state building and task_state spawning.
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.part --force-share --output=json" returned: 0 in 0.256s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.images [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] 74dfe7d1-36bb-426b-acac-1fe6efabb7b9 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.part /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Successfully created port: 612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441108.542807, 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] VM Resumed (Lifecycle Event)
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Instance spawned successfully.
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.part /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.converted" returned: 0 in 0.249s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441108.5623949, 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] VM Started (Lifecycle Event)
Apr 14 02:58:28 user nova-compute[71323]: INFO nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Took 9.80 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e.converted --force-share --output=json" returned: 0 in 0.310s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.285s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:29 user nova-compute[71323]: INFO nova.compute.manager [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Took 11.89 seconds to build instance.
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1b25eb11-e2eb-47b3-88da-e6dd18edfb21 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.252s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json" returned: 0 in 0.238s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.010s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json" returned: 0 in 0.252s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e,backing_fmt=raw /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e,backing_fmt=raw /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk 1073741824" returned: 0 in 0.107s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "2952a5f1d8e685bffd5079c6bcc24059df79fd6e" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.373s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Successfully updated port: 612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-changed-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Refreshing instance network info cache due to event network-changed-612e6a83-4a68-488b-85d3-c5144d53022c. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] Acquiring lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] Acquired lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.network.neutron [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Refreshing network info cache for port 612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e --force-share --output=json" returned: 0 in 0.260s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:29 user nova-compute[71323]: DEBUG nova.network.neutron [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Cannot resize image /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'migration_context' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Ensure instance console log exists: /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.network.neutron [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-763436ca-678b-47a8-a038-be02b66838bf req-49f2b12d-ad41-4bfc-bef2-50ba1140fd46 service nova] Releasing lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquired lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:58:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating instance_info_cache with network_info: [{"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Releasing lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance network_info: |[{"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Start _get_guest_xml network_info=[{"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'scsi', 'cdrom_bus': 'scsi', 'mapping': {'root': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'scsi', 'dev': 'sdb', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:58:16Z,direct_url=<?>,disk_format='qcow2',id=74dfe7d1-36bb-426b-acac-1fe6efabb7b9,min_disk=0,min_ram=0,name='',owner='13c210fe8b3645b4af91d1945a88effe',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:58:20Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/sda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'scsi', 'guest_format': None, 'device_name': '/dev/sda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '74dfe7d1-36bb-426b-acac-1fe6efabb7b9'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:58:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:58:16Z,direct_url=<?>,disk_format='qcow2',id=74dfe7d1-36bb-426b-acac-1fe6efabb7b9,min_disk=0,min_ram=0,name='',owner='13c210fe8b3645b4af91d1945a88effe',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:58:20Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T02:58:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-860564790',display_name='tempest-AttachSCSIVolumeTestJSON-server-860564790',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-860564790',id=8,image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDssy5+6+IRBX1/1QfIFiEyGurxnrzN7WU7XXH4VDLLVpl+RHIRY6CDaehyIbJeSPxvckSMHVq4L1xERiuNNi3d0vUlx65kRNHjlURIglNALBrmkJXBAu0neUdjagAUNcA==',key_name='tempest-keypair-1011448847',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bcbeb4e2fdc1420c98c264aac62340a7',ramdisk_id='',reservation_id='r-n6uvqa8r',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1798094222',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1798094222-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:28Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9eabf8254cde4d86a1862934ec9b5240',uuid=e712dfa7-67a4-4459-9446-94fd8785b09a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converting VIF {"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'pci_devices' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:58:31 user nova-compute[71323]:   <uuid>e712dfa7-67a4-4459-9446-94fd8785b09a</uuid>
Apr 14 02:58:31 user nova-compute[71323]:   <name>instance-00000008</name>
Apr 14 02:58:31 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:58:31 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:58:31 user nova-compute[71323]:   <metadata>
Apr 14 02:58:31 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:58:31 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:name>tempest-AttachSCSIVolumeTestJSON-server-860564790</nova:name>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:58:31</nova:creationTime>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:58:31 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:58:31 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:owner>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:user uuid="9eabf8254cde4d86a1862934ec9b5240">tempest-AttachSCSIVolumeTestJSON-1798094222-project-member</nova:user>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:project uuid="bcbeb4e2fdc1420c98c264aac62340a7">tempest-AttachSCSIVolumeTestJSON-1798094222</nova:project>
Apr 14 02:58:31 user nova-compute[71323]:       </nova:owner>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:root type="image" uuid="74dfe7d1-36bb-426b-acac-1fe6efabb7b9"/>
Apr 14 02:58:31 user nova-compute[71323]:       <nova:ports>
Apr 14 02:58:31 user nova-compute[71323]:         <nova:port uuid="612e6a83-4a68-488b-85d3-c5144d53022c">
Apr 14 02:58:31 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Apr 14 02:58:31 user nova-compute[71323]:         </nova:port>
Apr 14 02:58:31 user nova-compute[71323]:       </nova:ports>
Apr 14 02:58:31 user nova-compute[71323]:     </nova:instance>
Apr 14 02:58:31 user nova-compute[71323]:   </metadata>
Apr 14 02:58:31 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:58:31 user nova-compute[71323]:     <system>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="serial">e712dfa7-67a4-4459-9446-94fd8785b09a</entry>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="uuid">e712dfa7-67a4-4459-9446-94fd8785b09a</entry>
Apr 14 02:58:31 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:58:31 user nova-compute[71323]:     </system>
Apr 14 02:58:31 user nova-compute[71323]:   </sysinfo>
Apr 14 02:58:31 user nova-compute[71323]:   <os>
Apr 14 02:58:31 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:58:31 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:58:31 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:58:31 user nova-compute[71323]:   </os>
Apr 14 02:58:31 user nova-compute[71323]:   <features>
Apr 14 02:58:31 user nova-compute[71323]:     <acpi/>
Apr 14 02:58:31 user nova-compute[71323]:     <apic/>
Apr 14 02:58:31 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:58:31 user nova-compute[71323]:   </features>
Apr 14 02:58:31 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:58:31 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:58:31 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:58:31 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:58:31 user nova-compute[71323]:   </clock>
Apr 14 02:58:31 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:58:31 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:58:31 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:58:31 user nova-compute[71323]:   </cpu>
Apr 14 02:58:31 user nova-compute[71323]:   <devices>
Apr 14 02:58:31 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:58:31 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:58:31 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk"/>
Apr 14 02:58:31 user nova-compute[71323]:       <target dev="sda" bus="scsi"/>
Apr 14 02:58:31 user nova-compute[71323]:       <address type="drive" controller="0" unit="0"/>
Apr 14 02:58:31 user nova-compute[71323]:     </disk>
Apr 14 02:58:31 user nova-compute[71323]:     <disk type="file" device="cdrom">
Apr 14 02:58:31 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 02:58:31 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.config"/>
Apr 14 02:58:31 user nova-compute[71323]:       <target dev="sdb" bus="scsi"/>
Apr 14 02:58:31 user nova-compute[71323]:       <address type="drive" controller="0" unit="1"/>
Apr 14 02:58:31 user nova-compute[71323]:     </disk>
Apr 14 02:58:31 user nova-compute[71323]:     <controller type="scsi" index="0" model="virtio-scsi"/>
Apr 14 02:58:31 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:58:31 user nova-compute[71323]:       <mac address="fa:16:3e:fe:d0:2d"/>
Apr 14 02:58:31 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:31 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:58:31 user nova-compute[71323]:       <target dev="tap612e6a83-4a"/>
Apr 14 02:58:31 user nova-compute[71323]:     </interface>
Apr 14 02:58:31 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:58:31 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/console.log" append="off"/>
Apr 14 02:58:31 user nova-compute[71323]:     </serial>
Apr 14 02:58:31 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:58:31 user nova-compute[71323]:     <video>
Apr 14 02:58:31 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:58:31 user nova-compute[71323]:     </video>
Apr 14 02:58:31 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:58:31 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:58:31 user nova-compute[71323]:     </rng>
Apr 14 02:58:31 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:58:31 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:58:31 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:58:31 user nova-compute[71323]:     </memballoon>
Apr 14 02:58:31 user nova-compute[71323]:   </devices>
Apr 14 02:58:31 user nova-compute[71323]: </domain>
Apr 14 02:58:31 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T02:58:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-860564790',display_name='tempest-AttachSCSIVolumeTestJSON-server-860564790',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-860564790',id=8,image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDssy5+6+IRBX1/1QfIFiEyGurxnrzN7WU7XXH4VDLLVpl+RHIRY6CDaehyIbJeSPxvckSMHVq4L1xERiuNNi3d0vUlx65kRNHjlURIglNALBrmkJXBAu0neUdjagAUNcA==',key_name='tempest-keypair-1011448847',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bcbeb4e2fdc1420c98c264aac62340a7',ramdisk_id='',reservation_id='r-n6uvqa8r',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1798094222',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1798094222-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:28Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9eabf8254cde4d86a1862934ec9b5240',uuid=e712dfa7-67a4-4459-9446-94fd8785b09a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converting VIF {"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG os_vif [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap612e6a83-4a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap612e6a83-4a, col_values=(('external_ids', {'iface-id': '612e6a83-4a68-488b-85d3-c5144d53022c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fe:d0:2d', 'vm-uuid': 'e712dfa7-67a4-4459-9446-94fd8785b09a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:31 user nova-compute[71323]: INFO os_vif [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a')
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No BDM found with device name sda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No BDM found with device name sdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No VIF found with MAC fa:16:3e:fe:d0:2d, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:58:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Using config drive
Apr 14 02:58:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Creating config drive at /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.config
Apr 14 02:58:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpwwc4f7sl {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpwwc4f7sl" returned: 0 in 0.085s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] No waiting events found dispatching network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:35 user nova-compute[71323]: WARNING nova.compute.manager [req-28e79782-d4bb-4331-b882-86a78c1c6611 req-bd041173-c04a-494d-a28d-59bec0c99b9f service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received unexpected event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c for instance with vm_state building and task_state spawning.
Apr 14 02:58:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] No waiting events found dispatching network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:58:37 user nova-compute[71323]: WARNING nova.compute.manager [req-ef641b16-af5a-4857-8d28-44b1e29e9338 req-e28db039-f547-42b9-8de0-737535b616b2 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received unexpected event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c for instance with vm_state building and task_state spawning.
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441117.383879, e712dfa7-67a4-4459-9446-94fd8785b09a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] VM Resumed (Lifecycle Event)
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance spawned successfully.
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441117.3845944, e712dfa7-67a4-4459-9446-94fd8785b09a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] VM Started (Lifecycle Event)
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Took 9.85 seconds to spawn the instance on the hypervisor.
Apr 14 02:58:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:58:37 user nova-compute[71323]: INFO nova.compute.manager [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Took 11.18 seconds to build instance.
Apr 14 02:58:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-840bdffd-3183-425c-8c78-db9e4f34d51f tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.335s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 02:58:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 02:58:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updating instance_info_cache with network_info: [{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:58:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:58:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:58:52 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:52 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7731MB free_disk=26.73904800415039GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 82a9ec77-951b-4394-b555-00a49710acb5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance e712dfa7-67a4-4459-9446-94fd8785b09a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:58:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.793s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:58:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:58:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:58:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:59:24 user nova-compute[71323]: INFO nova.compute.claims [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Claim successful on node user
Apr 14 02:59:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.494s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:59:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.policy [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '337e2d8118324215b5a0baa36a5895b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6371b4c05df843da9b134e12994c1d7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:59:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Creating image(s)
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "/opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "/opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "/opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.187s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk 1073741824" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.336s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Successfully created port: 920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Cannot resize image /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'migration_context' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Ensure instance console log exists: /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Successfully updated port: 920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquired lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing instance network info cache due to event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] Acquiring lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating instance_info_cache with network_info: [{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Releasing lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Instance network_info: |[{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] Acquired lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Start _get_guest_xml network_info=[{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:59:27 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:27 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-491407941',display_name='tempest-TestStampPattern-server-491407941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-491407941',id=9,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cv7ox6d1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:26Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=2ca84990-b165-4d0b-a917-307cd8f96f28,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'pci_devices' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:59:27 user nova-compute[71323]:   <uuid>2ca84990-b165-4d0b-a917-307cd8f96f28</uuid>
Apr 14 02:59:27 user nova-compute[71323]:   <name>instance-00000009</name>
Apr 14 02:59:27 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:59:27 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:59:27 user nova-compute[71323]:   <metadata>
Apr 14 02:59:27 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:59:27 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:name>tempest-TestStampPattern-server-491407941</nova:name>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:59:27</nova:creationTime>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:59:27 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:59:27 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:owner>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:user uuid="337e2d8118324215b5a0baa36a5895b0">tempest-TestStampPattern-323884970-project-member</nova:user>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:project uuid="6371b4c05df843da9b134e12994c1d7a">tempest-TestStampPattern-323884970</nova:project>
Apr 14 02:59:27 user nova-compute[71323]:       </nova:owner>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:59:27 user nova-compute[71323]:       <nova:ports>
Apr 14 02:59:27 user nova-compute[71323]:         <nova:port uuid="920aad6f-1d46-48d5-81eb-95ffbc851da0">
Apr 14 02:59:27 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 02:59:27 user nova-compute[71323]:         </nova:port>
Apr 14 02:59:27 user nova-compute[71323]:       </nova:ports>
Apr 14 02:59:27 user nova-compute[71323]:     </nova:instance>
Apr 14 02:59:27 user nova-compute[71323]:   </metadata>
Apr 14 02:59:27 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:59:27 user nova-compute[71323]:     <system>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="serial">2ca84990-b165-4d0b-a917-307cd8f96f28</entry>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="uuid">2ca84990-b165-4d0b-a917-307cd8f96f28</entry>
Apr 14 02:59:27 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:59:27 user nova-compute[71323]:     </system>
Apr 14 02:59:27 user nova-compute[71323]:   </sysinfo>
Apr 14 02:59:27 user nova-compute[71323]:   <os>
Apr 14 02:59:27 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:59:27 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:59:27 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:59:27 user nova-compute[71323]:   </os>
Apr 14 02:59:27 user nova-compute[71323]:   <features>
Apr 14 02:59:27 user nova-compute[71323]:     <acpi/>
Apr 14 02:59:27 user nova-compute[71323]:     <apic/>
Apr 14 02:59:27 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:59:27 user nova-compute[71323]:   </features>
Apr 14 02:59:27 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:59:27 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:59:27 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:59:27 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:59:27 user nova-compute[71323]:   </clock>
Apr 14 02:59:27 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:59:27 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:59:27 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:59:27 user nova-compute[71323]:   </cpu>
Apr 14 02:59:27 user nova-compute[71323]:   <devices>
Apr 14 02:59:27 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:59:27 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:59:27 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk"/>
Apr 14 02:59:27 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:59:27 user nova-compute[71323]:     </disk>
Apr 14 02:59:27 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:59:27 user nova-compute[71323]:       <mac address="fa:16:3e:93:69:3c"/>
Apr 14 02:59:27 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:27 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:59:27 user nova-compute[71323]:       <target dev="tap920aad6f-1d"/>
Apr 14 02:59:27 user nova-compute[71323]:     </interface>
Apr 14 02:59:27 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:59:27 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/console.log" append="off"/>
Apr 14 02:59:27 user nova-compute[71323]:     </serial>
Apr 14 02:59:27 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:59:27 user nova-compute[71323]:     <video>
Apr 14 02:59:27 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:27 user nova-compute[71323]:     </video>
Apr 14 02:59:27 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:59:27 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:59:27 user nova-compute[71323]:     </rng>
Apr 14 02:59:27 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:59:27 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:59:27 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:59:27 user nova-compute[71323]:     </memballoon>
Apr 14 02:59:27 user nova-compute[71323]:   </devices>
Apr 14 02:59:27 user nova-compute[71323]: </domain>
Apr 14 02:59:27 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-491407941',display_name='tempest-TestStampPattern-server-491407941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-491407941',id=9,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cv7ox6d1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:26Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=2ca84990-b165-4d0b-a917-307cd8f96f28,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG os_vif [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap920aad6f-1d, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap920aad6f-1d, col_values=(('external_ids', {'iface-id': '920aad6f-1d46-48d5-81eb-95ffbc851da0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:93:69:3c', 'vm-uuid': '2ca84990-b165-4d0b-a917-307cd8f96f28'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:27 user nova-compute[71323]: INFO os_vif [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d')
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No VIF found with MAC fa:16:3e:93:69:3c, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:59:28 user nova-compute[71323]: DEBUG nova.network.neutron [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updated VIF entry in instance network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:59:28 user nova-compute[71323]: DEBUG nova.network.neutron [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating instance_info_cache with network_info: [{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9cf0afe2-bd11-4275-b7e9-8893068d3ac6 req-a32d7f69-2655-4696-b395-b942936304b2 service nova] Releasing lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] No waiting events found dispatching network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:29 user nova-compute[71323]: WARNING nova.compute.manager [req-45039b8f-f51a-456b-a6a2-c1945336cde3 req-d3374465-0ded-40f3-bf5f-7d5214450e6b service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received unexpected event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 for instance with vm_state building and task_state spawning.
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441171.531923, 2ca84990-b165-4d0b-a917-307cd8f96f28 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] VM Resumed (Lifecycle Event)
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Instance spawned successfully.
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441171.5332167, 2ca84990-b165-4d0b-a917-307cd8f96f28 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] VM Started (Lifecycle Event)
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Took 6.04 seconds to spawn the instance on the hypervisor.
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] No waiting events found dispatching network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:31 user nova-compute[71323]: WARNING nova.compute.manager [req-7e320338-b83b-40e9-a9c1-9391ba705ab5 req-056a94ab-6854-4851-a8ee-0fcb5f992c61 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received unexpected event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 for instance with vm_state building and task_state spawning.
Apr 14 02:59:31 user nova-compute[71323]: INFO nova.compute.manager [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Took 6.89 seconds to build instance.
Apr 14 02:59:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6a8650fb-a5d9-46c4-8163-dee963af159b tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:59:37 user nova-compute[71323]: INFO nova.compute.claims [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Claim successful on node user
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.362s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:59:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:59:37 user nova-compute[71323]: INFO nova.virt.block_device [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Booting with volume snapshot ce95f5fe-2669-423c-8ac4-9edd6f1c5f54 at /dev/vda
Apr 14 02:59:37 user nova-compute[71323]: DEBUG nova.policy [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:59:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Successfully created port: c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Successfully updated port: c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:38 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-changed-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Refreshing instance network info cache due to event network-changed-c75de928-a52a-4a2a-b413-15a2be115846. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] Acquiring lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Updating instance_info_cache with network_info: [{"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Instance network_info: |[{"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] Acquired lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.network.neutron [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Refreshing network info cache for port c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.network.neutron [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Updated VIF entry in instance network info cache for port c75de928-a52a-4a2a-b413-15a2be115846. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG nova.network.neutron [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Updating instance_info_cache with network_info: [{"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b4a5c682-3e37-4847-85b2-a59609a9802c req-ff70c618-6f83-4a61-8644-8b9d73fbba26 service nova] Releasing lock "refresh_cache-bc99f782-81c1-4243-8853-5554e5663d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 02:59:41 user nova-compute[71323]: INFO nova.compute.claims [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Claim successful on node user
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.373s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 02:59:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.policy [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0dbce21e767a4a61b9213e38523d247e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f021342634dd4f16bf2720533070154c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:59:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Creating image(s)
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "/opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "/opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "/opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk 1073741824" returned: 0 in 0.046s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.206s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Successfully created port: 8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Cannot resize image /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.objects.instance [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'migration_context' on Instance uuid 8a826438-d96e-4e70-b50e-aca4be008394 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Ensure instance console log exists: /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Successfully updated port: 8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquired lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-changed-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Refreshing instance network info cache due to event network-changed-8151f818-904f-472f-b66c-ee41b2c2975e. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] Acquiring lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG os_brick.utils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 02:59:43 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpws9qkls9/privsep.sock']
Apr 14 02:59:43 user sudo[81975]:    stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpws9qkls9/privsep.sock
Apr 14 02:59:43 user sudo[81975]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Updating instance_info_cache with network_info: [{"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Releasing lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Instance network_info: |[{"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] Acquired lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Refreshing network info cache for port 8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Start _get_guest_xml network_info=[{"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:59:43 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:43 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-408869365',display_name='tempest-ServersNegativeTestJSON-server-408869365',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-408869365',id=11,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-clw8zz05',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:42Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=8a826438-d96e-4e70-b50e-aca4be008394,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.objects.instance [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'pci_devices' on Instance uuid 8a826438-d96e-4e70-b50e-aca4be008394 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:59:43 user nova-compute[71323]:   <uuid>8a826438-d96e-4e70-b50e-aca4be008394</uuid>
Apr 14 02:59:43 user nova-compute[71323]:   <name>instance-0000000b</name>
Apr 14 02:59:43 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:59:43 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:59:43 user nova-compute[71323]:   <metadata>
Apr 14 02:59:43 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:59:43 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:name>tempest-ServersNegativeTestJSON-server-408869365</nova:name>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:59:43</nova:creationTime>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:59:43 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:59:43 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:owner>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:user uuid="0dbce21e767a4a61b9213e38523d247e">tempest-ServersNegativeTestJSON-675126189-project-member</nova:user>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:project uuid="f021342634dd4f16bf2720533070154c">tempest-ServersNegativeTestJSON-675126189</nova:project>
Apr 14 02:59:43 user nova-compute[71323]:       </nova:owner>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:59:43 user nova-compute[71323]:       <nova:ports>
Apr 14 02:59:43 user nova-compute[71323]:         <nova:port uuid="8151f818-904f-472f-b66c-ee41b2c2975e">
Apr 14 02:59:43 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 02:59:43 user nova-compute[71323]:         </nova:port>
Apr 14 02:59:43 user nova-compute[71323]:       </nova:ports>
Apr 14 02:59:43 user nova-compute[71323]:     </nova:instance>
Apr 14 02:59:43 user nova-compute[71323]:   </metadata>
Apr 14 02:59:43 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:59:43 user nova-compute[71323]:     <system>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="serial">8a826438-d96e-4e70-b50e-aca4be008394</entry>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="uuid">8a826438-d96e-4e70-b50e-aca4be008394</entry>
Apr 14 02:59:43 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:59:43 user nova-compute[71323]:     </system>
Apr 14 02:59:43 user nova-compute[71323]:   </sysinfo>
Apr 14 02:59:43 user nova-compute[71323]:   <os>
Apr 14 02:59:43 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:59:43 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:59:43 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:59:43 user nova-compute[71323]:   </os>
Apr 14 02:59:43 user nova-compute[71323]:   <features>
Apr 14 02:59:43 user nova-compute[71323]:     <acpi/>
Apr 14 02:59:43 user nova-compute[71323]:     <apic/>
Apr 14 02:59:43 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:59:43 user nova-compute[71323]:   </features>
Apr 14 02:59:43 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:59:43 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:59:43 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:59:43 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:59:43 user nova-compute[71323]:   </clock>
Apr 14 02:59:43 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:59:43 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:59:43 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:59:43 user nova-compute[71323]:   </cpu>
Apr 14 02:59:43 user nova-compute[71323]:   <devices>
Apr 14 02:59:43 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:59:43 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:59:43 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk"/>
Apr 14 02:59:43 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:59:43 user nova-compute[71323]:     </disk>
Apr 14 02:59:43 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:59:43 user nova-compute[71323]:       <mac address="fa:16:3e:aa:18:3d"/>
Apr 14 02:59:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:43 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:59:43 user nova-compute[71323]:       <target dev="tap8151f818-90"/>
Apr 14 02:59:43 user nova-compute[71323]:     </interface>
Apr 14 02:59:43 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:59:43 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/console.log" append="off"/>
Apr 14 02:59:43 user nova-compute[71323]:     </serial>
Apr 14 02:59:43 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:59:43 user nova-compute[71323]:     <video>
Apr 14 02:59:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:43 user nova-compute[71323]:     </video>
Apr 14 02:59:43 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:59:43 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:59:43 user nova-compute[71323]:     </rng>
Apr 14 02:59:43 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:59:43 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:59:43 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:59:43 user nova-compute[71323]:     </memballoon>
Apr 14 02:59:43 user nova-compute[71323]:   </devices>
Apr 14 02:59:43 user nova-compute[71323]: </domain>
Apr 14 02:59:43 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-408869365',display_name='tempest-ServersNegativeTestJSON-server-408869365',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-408869365',id=11,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-clw8zz05',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:42Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=8a826438-d96e-4e70-b50e-aca4be008394,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG os_vif [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8151f818-90, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8151f818-90, col_values=(('external_ids', {'iface-id': '8151f818-904f-472f-b66c-ee41b2c2975e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:aa:18:3d', 'vm-uuid': '8a826438-d96e-4e70-b50e-aca4be008394'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:43 user nova-compute[71323]: INFO os_vif [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90')
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] No VIF found with MAC fa:16:3e:aa:18:3d, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Updated VIF entry in instance network info cache for port 8151f818-904f-472f-b66c-ee41b2c2975e. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:59:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Updating instance_info_cache with network_info: [{"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d47672aa-509d-4016-a393-bc64e0513863 req-eb2329f2-60a7-40aa-9cf2-060e5b4c9172 service nova] Releasing lock "refresh_cache-8a826438-d96e-4e70-b50e-aca4be008394" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:44 user sudo[81975]: pam_unix(sudo:session): session closed for user root
Apr 14 02:59:44 user nova-compute[71323]: INFO oslo.privsep.daemon [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Spawned new privsep daemon via rootwrap
Apr 14 02:59:44 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon starting
Apr 14 02:59:44 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Apr 14 02:59:44 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Apr 14 02:59:44 user nova-compute[71323]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 81992
Apr 14 02:59:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5f32f8f-2793-4606-a583-167ab2759879]: (2,) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a138fa8c-9894-44ab-8d7f-654f67240a03]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.nvmeof [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath'
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbe1dbac-f2bf-4f6d-920d-ca1878614dcb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9bf8b0e-655d-48c7-8932-734d8740e8a1]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[010955fb-6375-4501-8558-4268f8f26214]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:45 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[89d8fc5e-cd90-4d7b-88e1-877b7637df33]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG os_brick.utils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (1911ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Updating existing volume attachment record: 9a5f6596-6d83-438b-bfae-fe7d1c5b35bf {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] No waiting events found dispatching network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:45 user nova-compute[71323]: WARNING nova.compute.manager [req-f439534e-ab85-4c01-b0f0-5595a69454d6 req-c54e2595-7065-4b50-a1fc-a11d146dbd62 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received unexpected event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e for instance with vm_state building and task_state spawning.
Apr 14 02:59:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-changed-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Refreshing instance network info cache due to event network-changed-b85a839f-973d-415e-94b6-cdb480a8b89d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:59:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Refreshing network info cache for port b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441187.1695008, 8a826438-d96e-4e70-b50e-aca4be008394 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] VM Resumed (Lifecycle Event)
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Instance spawned successfully.
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441187.175431, 8a826438-d96e-4e70-b50e-aca4be008394 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] VM Started (Lifecycle Event)
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Took 5.36 seconds to spawn the instance on the hypervisor.
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.network.neutron [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updated VIF entry in instance network info cache for port b85a839f-973d-415e-94b6-cdb480a8b89d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.network.neutron [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39750a2e-6a0d-432a-876f-6ca762b7038d req-c8239cfc-527b-423e-a6a0-1a469d2568b2 service nova] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Took 6.04 seconds to build instance.
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0ebbf1ee-3414-48a7-9e38-99e3fc9bd8cf tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.134s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] No waiting events found dispatching network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:47 user nova-compute[71323]: WARNING nova.compute.manager [req-82294257-372c-403b-94c6-5ee0f11baf24 req-a9be5deb-2ef8-4393-a4e6-570bd9f50000 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received unexpected event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e for instance with vm_state active and task_state None.
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:47 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:48 user nova-compute[71323]: INFO nova.compute.manager [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Rescuing
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:49 user nova-compute[71323]: WARNING nova.compute.manager [req-96bef6af-199b-4676-9de3-3a69f03acca1 req-d1203f7e-39e4-4184-a25c-8c9ab0ee1ab0 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state active and task_state rescuing.
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.098s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.compute.manager [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Attaching volume d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 to /dev/vdb
Apr 14 02:59:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.utils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Creating image(s)
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Ensure instance console log exists: /opt/stack/data/nova/instances/bc99f782-81c1-4243-8853-5554e5663d0a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Start _get_guest_xml network_info=[{"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '9a5f6596-6d83-438b-bfae-fe7d1c5b35bf', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JcvzNKnb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'bc99f782-81c1-4243-8853-5554e5663d0a', 'attached_at': '', 'detached_at': '', 'volume_id': '47e890f1-6257-486b-ab68-7d91ac9f65bf', 'serial': '47e890f1-6257-486b-ab68-7d91ac9f65bf'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2898a9bb-f1e3-4b77-a8a4-2e5877db163a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "cache_volume_driver" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "cache_volume_driver" acquired by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connector [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Factory for ISCSI on None {{(pid=71323) factory /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connector.py:281}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "cache_volume_driver" "released" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: held 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JcvzNKnb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 02:59:50 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[008e84d3-8d4e-4d96-a155-2cd1912168e4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1298f8b0-6bdb-44b4-918c-2a78f4965f60]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 02:59:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d124f3c-31bf-48e7-ae24-09328ffd043b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d13b3a05-b2de-42a4-ba3f-f4829de638a9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b767374-3647-45c6-ac23-2f7c8a68b3ad]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf] added
Apr 14 02:59:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d80d1d7a-72ba-40c1-a655-54f0976ed626]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.utils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== get_connector_properties: return (176ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating existing volume attachment record: 74d9f89b-3416-45c5-8af8-870134e07548 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf8406db-8b9b-4dcd-a35a-f1dfc849baba]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance destroyed successfully.
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Attempting rescue
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Creating image(s)
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'trusted_certs' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[769f93ba-50ef-436f-bf58-1bcd2af09010]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JcvzNKnb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JcvzNKnb" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b02b084-0cf7-44da-ad7e-c72239546bdf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'JcvzNKnb'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bea5ba9-8647-4442-a308-2aabc6d4712f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e99495c-5a9b-4df2-83e3-ffd4ce619636]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 02:59:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 02:59:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 02:59:50 user nova-compute[71323]: 
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2141e731-23e9-4f9b-8572-cca81c7c33b1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260]
Apr 14 02:59:50 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260] successful.
Apr 14 02:59:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[915f05ae-ea35-420a-afbe-370c719f9537]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0135fe3-e436-4079-91ae-65180c503b77]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue" returned: 0 in 0.080s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.427s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('33', '-', '-', 0) found on session 2 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.242s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start _get_guest_xml network_info=[{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:b2:39:e3"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3d76a17-6b73-4cac-92d7-56df11ef782e]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'numa_topology' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'vcpu_model' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1086696024',display_name='tempest-ServerRescueNegativeTestJSON-server-1086696024',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1086696024',id=4,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-d1yclxnx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:14Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=ac032176-63bc-457c-ad53-5b4a854eec34,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:b2:39:e3"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:b2:39:e3"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:59:50 user nova-compute[71323]:   <uuid>ac032176-63bc-457c-ad53-5b4a854eec34</uuid>
Apr 14 02:59:50 user nova-compute[71323]:   <name>instance-00000004</name>
Apr 14 02:59:50 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:59:50 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:59:50 user nova-compute[71323]:   <metadata>
Apr 14 02:59:50 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:59:50 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-1086696024</nova:name>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:59:50</nova:creationTime>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:59:50 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:59:50 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:owner>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 02:59:50 user nova-compute[71323]:       </nova:owner>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 02:59:50 user nova-compute[71323]:       <nova:ports>
Apr 14 02:59:50 user nova-compute[71323]:         <nova:port uuid="59fef74f-8c76-4dd5-bc99-311eea495950">
Apr 14 02:59:50 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 02:59:50 user nova-compute[71323]:         </nova:port>
Apr 14 02:59:50 user nova-compute[71323]:       </nova:ports>
Apr 14 02:59:50 user nova-compute[71323]:     </nova:instance>
Apr 14 02:59:50 user nova-compute[71323]:   </metadata>
Apr 14 02:59:50 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:59:50 user nova-compute[71323]:     <system>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="serial">ac032176-63bc-457c-ad53-5b4a854eec34</entry>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="uuid">ac032176-63bc-457c-ad53-5b4a854eec34</entry>
Apr 14 02:59:50 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:59:50 user nova-compute[71323]:     </system>
Apr 14 02:59:50 user nova-compute[71323]:   </sysinfo>
Apr 14 02:59:50 user nova-compute[71323]:   <os>
Apr 14 02:59:50 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:59:50 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:59:50 user nova-compute[71323]:   </os>
Apr 14 02:59:50 user nova-compute[71323]:   <features>
Apr 14 02:59:50 user nova-compute[71323]:     <acpi/>
Apr 14 02:59:50 user nova-compute[71323]:     <apic/>
Apr 14 02:59:50 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:59:50 user nova-compute[71323]:   </features>
Apr 14 02:59:50 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:59:50 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:59:50 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:59:50 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:59:50 user nova-compute[71323]:   </clock>
Apr 14 02:59:50 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:59:50 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:59:50 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:59:50 user nova-compute[71323]:   </cpu>
Apr 14 02:59:50 user nova-compute[71323]:   <devices>
Apr 14 02:59:50 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:59:50 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:59:50 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue"/>
Apr 14 02:59:50 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:59:50 user nova-compute[71323]:     </disk>
Apr 14 02:59:50 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 02:59:50 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 02:59:50 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk"/>
Apr 14 02:59:50 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 02:59:50 user nova-compute[71323]:     </disk>
Apr 14 02:59:50 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:59:50 user nova-compute[71323]:       <mac address="fa:16:3e:b2:39:e3"/>
Apr 14 02:59:50 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:50 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:59:50 user nova-compute[71323]:       <target dev="tap59fef74f-8c"/>
Apr 14 02:59:50 user nova-compute[71323]:     </interface>
Apr 14 02:59:50 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:59:50 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/console.log" append="off"/>
Apr 14 02:59:50 user nova-compute[71323]:     </serial>
Apr 14 02:59:50 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:59:50 user nova-compute[71323]:     <video>
Apr 14 02:59:50 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:50 user nova-compute[71323]:     </video>
Apr 14 02:59:50 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:59:50 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:59:50 user nova-compute[71323]:     </rng>
Apr 14 02:59:50 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:59:50 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:59:50 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:59:50 user nova-compute[71323]:     </memballoon>
Apr 14 02:59:50 user nova-compute[71323]:   </devices>
Apr 14 02:59:50 user nova-compute[71323]: </domain>
Apr 14 02:59:50 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:59:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance destroyed successfully.
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:b2:39:e3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JcvzNKnb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.470s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1471ms) {'type': 'block', 'scsi_wwn': '23861323933633364', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23861323933633364', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-changed-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Refreshing instance network info cache due to event network-changed-04b5c1f1-8988-4a34-b97a-ad0cecd12b71. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Refreshing network info cache for port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-432581934',display_name='tempest-TestVolumeBootPattern-server-432581934',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-432581934',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-fnw79iau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:38Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=bc99f782-81c1-4243-8853-5554e5663d0a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.objects.instance [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid bc99f782-81c1-4243-8853-5554e5663d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 02:59:51 user nova-compute[71323]:   <uuid>bc99f782-81c1-4243-8853-5554e5663d0a</uuid>
Apr 14 02:59:51 user nova-compute[71323]:   <name>instance-0000000a</name>
Apr 14 02:59:51 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 02:59:51 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 02:59:51 user nova-compute[71323]:   <metadata>
Apr 14 02:59:51 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 02:59:51 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 02:59:51 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-server-432581934</nova:name>
Apr 14 02:59:51 user nova-compute[71323]:       <nova:creationTime>2023-04-14 02:59:50</nova:creationTime>
Apr 14 02:59:51 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 02:59:51 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 02:59:51 user nova-compute[71323]:       </nova:flavor>
Apr 14 02:59:51 user nova-compute[71323]:       <nova:owner>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 02:59:51 user nova-compute[71323]:       </nova:owner>
Apr 14 02:59:51 user nova-compute[71323]:       <nova:ports>
Apr 14 02:59:51 user nova-compute[71323]:         <nova:port uuid="c75de928-a52a-4a2a-b413-15a2be115846">
Apr 14 02:59:51 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 02:59:51 user nova-compute[71323]:         </nova:port>
Apr 14 02:59:51 user nova-compute[71323]:       </nova:ports>
Apr 14 02:59:51 user nova-compute[71323]:     </nova:instance>
Apr 14 02:59:51 user nova-compute[71323]:   </metadata>
Apr 14 02:59:51 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 02:59:51 user nova-compute[71323]:     <system>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="serial">bc99f782-81c1-4243-8853-5554e5663d0a</entry>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="uuid">bc99f782-81c1-4243-8853-5554e5663d0a</entry>
Apr 14 02:59:51 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 02:59:51 user nova-compute[71323]:     </system>
Apr 14 02:59:51 user nova-compute[71323]:   </sysinfo>
Apr 14 02:59:51 user nova-compute[71323]:   <os>
Apr 14 02:59:51 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 02:59:51 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 02:59:51 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 02:59:51 user nova-compute[71323]:   </os>
Apr 14 02:59:51 user nova-compute[71323]:   <features>
Apr 14 02:59:51 user nova-compute[71323]:     <acpi/>
Apr 14 02:59:51 user nova-compute[71323]:     <apic/>
Apr 14 02:59:51 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 02:59:51 user nova-compute[71323]:   </features>
Apr 14 02:59:51 user nova-compute[71323]:   <clock offset="utc">
Apr 14 02:59:51 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 02:59:51 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 02:59:51 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 02:59:51 user nova-compute[71323]:   </clock>
Apr 14 02:59:51 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 02:59:51 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 02:59:51 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 02:59:51 user nova-compute[71323]:   </cpu>
Apr 14 02:59:51 user nova-compute[71323]:   <devices>
Apr 14 02:59:51 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 02:59:51 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 02:59:51 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 02:59:51 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 02:59:51 user nova-compute[71323]:       <serial>47e890f1-6257-486b-ab68-7d91ac9f65bf</serial>
Apr 14 02:59:51 user nova-compute[71323]:     </disk>
Apr 14 02:59:51 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 02:59:51 user nova-compute[71323]:       <mac address="fa:16:3e:79:f5:42"/>
Apr 14 02:59:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:51 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 02:59:51 user nova-compute[71323]:       <target dev="tapc75de928-a5"/>
Apr 14 02:59:51 user nova-compute[71323]:     </interface>
Apr 14 02:59:51 user nova-compute[71323]:     <serial type="pty">
Apr 14 02:59:51 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bc99f782-81c1-4243-8853-5554e5663d0a/console.log" append="off"/>
Apr 14 02:59:51 user nova-compute[71323]:     </serial>
Apr 14 02:59:51 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 02:59:51 user nova-compute[71323]:     <video>
Apr 14 02:59:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 02:59:51 user nova-compute[71323]:     </video>
Apr 14 02:59:51 user nova-compute[71323]:     <rng model="virtio">
Apr 14 02:59:51 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 02:59:51 user nova-compute[71323]:     </rng>
Apr 14 02:59:51 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 02:59:51 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 02:59:51 user nova-compute[71323]:       <stats period="10"/>
Apr 14 02:59:51 user nova-compute[71323]:     </memballoon>
Apr 14 02:59:51 user nova-compute[71323]:   </devices>
Apr 14 02:59:51 user nova-compute[71323]: </domain>
Apr 14 02:59:51 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-432581934',display_name='tempest-TestVolumeBootPattern-server-432581934',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-432581934',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-fnw79iau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:38Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=bc99f782-81c1-4243-8853-5554e5663d0a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG os_vif [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc75de928-a5, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc75de928-a5, col_values=(('external_ids', {'iface-id': 'c75de928-a52a-4a2a-b413-15a2be115846', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:79:f5:42', 'vm-uuid': 'bc99f782-81c1-4243-8853-5554e5663d0a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:51 user nova-compute[71323]: INFO os_vif [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5')
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:51 user nova-compute[71323]: WARNING nova.compute.manager [req-a8cdeb0a-8622-4d7b-ae06-25760352a632 req-06e4be0e-2068-45c5-8513-2cdb4d3bd205 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state active and task_state rescuing.
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:79:f5:42, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updated VIF entry in instance network info cache for port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98c39dda-0590-486a-b062-7efbdc16b2bc req-d7074c5a-ebb9-4fac-902e-badd3efb0fe1 service nova] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:54 user nova-compute[71323]: WARNING nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state active and task_state rescuing.
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:54 user nova-compute[71323]: WARNING nova.compute.manager [req-5fc3e0c8-3778-492f-a08c-8b79a26ca93d req-013a0635-9264-4fb2-91cb-2984f37cfd01 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state active and task_state rescuing.
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.201s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] No waiting events found dispatching network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:54 user nova-compute[71323]: WARNING nova.compute.manager [req-29cccbc5-c365-4665-986d-162e32a3f271 req-fbf0880b-e131-4213-a3fd-16dc6ab86373 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received unexpected event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 for instance with vm_state building and task_state spawning.
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 02:59:55 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 02:59:55 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 02:59:55 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1093c8d3-4494-461c-9636-af849035ff06]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 02:59:55 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d42d02bb-8568-4fda-b313-94788eaaa59b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8] added
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[55a055fe-f13c-4601-97ce-e505ba2cfca2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7822d64b-689f-434c-9fea-61b116daa3b3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mpkibDRE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mpkibDRE" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a926588e-22c4-47db-bda4-75b35bb02f8d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mpkibDRE'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9028cc58-9a6d-4c36-ad5b-5a247805be0c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f90023f-135f-4f0b-ba70-65f9c3659892]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1f7b507-6ae4-42a8-9584-acbd34d71f62]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]
Apr 14 02:59:55 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ad215f6-5794-468a-b608-1d296f8d61de]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[398d85a9-d3bc-4fbb-aea6-bdf26e063f06]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:55 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 02:59:55 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 02:59:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.363s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] HCTL ('34', '-', '-', 0) found on session 3 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[33c19223-b143-4a57-bc36-ce76b7a24374]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Searching for a device in session 3 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 02:59:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] No waiting events found dispatching network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:56 user nova-compute[71323]: WARNING nova.compute.manager [req-1ad25374-77a2-4f17-960f-bfdc192b4dbc req-a7139a54-8ee7-4e1e-a865-3b76aed06c6c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received unexpected event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 for instance with vm_state building and task_state spawning.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for ac032176-63bc-457c-ad53-5b4a854eec34 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441195.8502183, ac032176-63bc-457c-ad53-5b4a854eec34 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] VM Resumed (Lifecycle Event)
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-90a19a10-8af2-42f6-965a-65441998fb19 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:56 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7835MB free_disk=26.683792114257812GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Instance spawned successfully.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441195.8503656, ac032176-63bc-457c-ad53-5b4a854eec34 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] VM Started (Lifecycle Event)
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.399s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== connect_volume: return (1401ms) {'type': 'block', 'scsi_wwn': '23531323766386562', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23531323766386562', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 02:59:56 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 02:59:56 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 02:59:56 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 02:59:56 user nova-compute[71323]:   <serial>d1ac2d2b-c18f-4958-a86e-fa298a15c9b8</serial>
Apr 14 02:59:56 user nova-compute[71323]: </disk>
Apr 14 02:59:56 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Took 6.67 seconds to spawn the instance on the hypervisor.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441196.2375557, bc99f782-81c1-4243-8853-5554e5663d0a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] VM Resumed (Lifecycle Event)
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc99f782-81c1-4243-8853-5554e5663d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 82a9ec77-951b-4394-b555-00a49710acb5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance e712dfa7-67a4-4459-9446-94fd8785b09a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a826438-d96e-4e70-b50e-aca4be008394 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441196.2391715, bc99f782-81c1-4243-8853-5554e5663d0a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] VM Started (Lifecycle Event)
Apr 14 02:59:56 user nova-compute[71323]: INFO nova.compute.manager [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Took 19.92 seconds to build instance.
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 02:59:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-99f03fd2-5b9b-4316-9e40-7059ed19e510 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 20.036s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No VIF found with MAC fa:16:3e:f7:89:a6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.718s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9e23dcc2-0c6c-401f-b8e5-3301583b9b7c tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.324s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG nova.objects.instance [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lazy-loading 'flavor' on Instance uuid 82a9ec77-951b-4394-b555-00a49710acb5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.061s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:58 user nova-compute[71323]: INFO nova.compute.manager [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Attaching volume d9358de3-07e9-437f-983f-50a585699a89 to /dev/vdb
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.utils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[43151aec-775f-4be2-b916-85ef6f227c8b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=71323) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1b4d97f-c5f0-405a-8e6b-309b06f81a2a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5026d16-23fa-41d5-9926-3f4ca96b67f9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3d4be5a-d943-454b-a722-94e2701dd9be]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 02:59:58 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8739403-2332-477b-a763-7c87a2aeadf4]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG os_brick.utils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Updating existing volume attachment record: ce6ca826-b762-4f71-9690-52f856d57d91 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 02:59:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 02:59:58 user nova-compute[71323]: WARNING nova.compute.manager [req-52eb71fd-f7f2-46e4-8203-b36982044a9c req-9cdc589f-f0ec-46cf-ac00-da6984b26afa service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state powering-off.
Apr 14 02:59:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance destroyed successfully.
Apr 14 02:59:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 02:59:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b68fdddc-d2b1-4904-9ab5-d2c77f320d8b tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 1.081s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:00 user nova-compute[71323]: WARNING nova.compute.manager [req-b8371578-3159-4591-9446-a77dcbe4d746 req-2c19be0d-5111-4d14-85fa-f30c05b696e1 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state stopped and task_state None.
Apr 14 03:00:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'noqgYikX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:02 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:02 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:02 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0ae81e8-7a88-4b8b-99de-10187ba45a46]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:00:02 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7733c4bf-63ca-4a09-a314-bd5e2996c71c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89] added
Apr 14 03:00:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a84713e-e802-4f9d-aacc-3f3262a61306]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ea635b0-ec35-4e58-9376-209bd2266077]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v noqgYikX {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v noqgYikX" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1ea6c90-9d32-4fb6-a6ea-d64b532f46f0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'noqgYikX'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f270fc5-f418-47a8-947e-75f02a197232]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f563b664-7fc8-4eb1-a7b4-13e375b94b62]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:02 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:02 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[27ada439-b6d2-46b6-9a6a-ef3533b47864]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260]
Apr 14 03:00:03 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa4e4e07-27ae-4ea0-aa42-73361962a0fa]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3df0d728-48c6-45ad-bdb7-9e41292380e2]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:03 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:03 user nova-compute[71323]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)
Apr 14 03:00:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:03 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:03 user nova-compute[71323]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)
Apr 14 03:00:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] HCTL ('35', '-', '-', 0) found on session 4 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9dbf6af-506f-4094-850e-8a2a2801b917]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'noqgYikX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:00:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.306s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] <== connect_volume: return (1307ms) {'type': 'block', 'scsi_wwn': '23834326637306364', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23834326637306364', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lazy-loading 'flavor' on Instance uuid 82a9ec77-951b-4394-b555-00a49710acb5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:00:04 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:04 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:04 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:00:04 user nova-compute[71323]:   <serial>d9358de3-07e9-437f-983f-50a585699a89</serial>
Apr 14 03:00:04 user nova-compute[71323]: </disk>
Apr 14 03:00:04 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] No VIF found with MAC fa:16:3e:13:c8:bf, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:00:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-982938a8-b635-4c33-bd57-6c2111a7dbb9 tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.738s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:05 user nova-compute[71323]: INFO nova.compute.manager [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Terminating instance
Apr 14 03:00:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-a54adc6f-35a2-48cd-9555-487efac507d0 req-f9b2d367-4f59-471c-afd6-3d763ecf41c4 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Instance destroyed successfully.
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.objects.instance [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lazy-loading 'resources' on Instance uuid 82a9ec77-951b-4394-b555-00a49710acb5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-815548039',display_name='tempest-DeleteServersTestJSON-server-815548039',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-815548039',id=6,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='46cec85cd4364c8eaad6870b92249a56',ramdisk_id='',reservation_id='r-is0fgfby',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-DeleteServersTestJSON-360109579',owner_user_name='tempest-DeleteServersTestJSON-360109579-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:58:17Z,user_data=None,user_id='70eb1e59ba1342688a8ce4e6ff440f58',uuid=82a9ec77-951b-4394-b555-00a49710acb5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converting VIF {"id": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "address": "fa:16:3e:13:c8:bf", "network": {"id": "19fce8c0-8eeb-41ff-a4b8-6a7a14163b21", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-1425513378-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46cec85cd4364c8eaad6870b92249a56", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap35c25a8a-14", "ovs_interfaceid": "35c25a8a-1450-4b7a-8a1f-11c047bb9d27", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_vif [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap35c25a8a-14, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: INFO os_vif [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:13:c8:bf,bridge_name='br-int',has_traffic_filtering=True,id=35c25a8a-1450-4b7a-8a1f-11c047bb9d27,network=Network(19fce8c0-8eeb-41ff-a4b8-6a7a14163b21),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap35c25a8a-14')
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'noqgYikX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:06 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[385d7cfb-341e-4b73-be0d-e0bb67111691]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:00:06 user nova-compute[71323]: No targets found.
Apr 14 03:00:06 user nova-compute[71323]: iSNS:
Apr 14 03:00:06 user nova-compute[71323]: No targets found.
Apr 14 03:00:06 user nova-compute[71323]: STATIC:
Apr 14 03:00:06 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89
Apr 14 03:00:06 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:06 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:06 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf
Apr 14 03:00:06 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:06 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:06 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:00:06 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:06 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:06 user nova-compute[71323]: FIRMWARE:
Apr 14 03:00:06 user nova-compute[71323]: No targets found.
Apr 14 03:00:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:00:06 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:00:06 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bc6a3ac-2998-4ae4-b4f7-9533244363e0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.044s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[35d1dc7e-bc30-4940-8018-c123f0eafe03]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:06 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:06 user nova-compute[71323]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)
Apr 14 03:00:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:06 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:06 user nova-compute[71323]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 (non-flash)
Apr 14 03:00:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cda12d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b5d22a4-4a3d-4683-8cfa-7e3fd8e74d9f]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1da8a79-3500-4b43-985c-cd4d8ad3b443]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fc8815e-9935-47dc-b55a-7eb0fc990675]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8118103c-c3f6-4d67-90d7-e6a18bcba2d2]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[95f7a8fb-0db5-42b7-a101-a9db83ae49b4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --logout" returned: 0 in 0.080s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cff48db3-57d3-42d6-b0c3-17f07eca4d4c]: (4, ('Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260]\nLogout of [sid: 4, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260]
Apr 14 03:00:06 user nova-compute[71323]: Logout of [sid: 4, target: iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d9358de3-07e9-437f-983f-50a585699a89 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ab5f698-63c0-40fc-895b-b662a3d0eba9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.388s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] <== disconnect_volume: return (390ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:00:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Deleting instance files /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5_del
Apr 14 03:00:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Deletion of /opt/stack/data/nova/instances/82a9ec77-951b-4394-b555-00a49710acb5_del complete
Apr 14 03:00:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Checking UEFI support for host arch (x86_64) {{(pid=71323) supports_uefi /opt/stack/nova/nova/virt/libvirt/host.py:1722}}
Apr 14 03:00:07 user nova-compute[71323]: INFO nova.virt.libvirt.host [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] UEFI support detected
Apr 14 03:00:07 user nova-compute[71323]: INFO nova.compute.manager [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Took 1.29 seconds to destroy the instance on the hypervisor.
Apr 14 03:00:07 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:08 user nova-compute[71323]: WARNING nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state active and task_state deleting.
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:08 user nova-compute[71323]: WARNING nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state active and task_state deleting.
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:08 user nova-compute[71323]: WARNING nova.compute.manager [req-48521572-930d-4d99-b7b1-6b8162e18526 req-6c2a3cdd-d8c5-4948-b35c-a59cd92dcab1 service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state active and task_state deleting.
Apr 14 03:00:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:08 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:08 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Took 1.72 seconds to deallocate network for instance.
Apr 14 03:00:09 user nova-compute[71323]: INFO nova.compute.manager [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Took 1.15 seconds to detach 1 volumes for instance.
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:10 user nova-compute[71323]: WARNING nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-unplugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state deleted and task_state None.
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-changed-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Refreshing instance network info cache due to event network-changed-9a41964d-ca86-41d7-8287-1af6e94403b4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Acquiring lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Acquired lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Refreshing network info cache for port 9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.371s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:10 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Deleted allocations for instance 82a9ec77-951b-4394-b555-00a49710acb5
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-925d4cca-a93c-4e3c-ab52-b34df0fbfd7f tempest-DeleteServersTestJSON-360109579 tempest-DeleteServersTestJSON-360109579-project-member] Lock "82a9ec77-951b-4394-b555-00a49710acb5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.838s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updated VIF entry in instance network info cache for port 9a41964d-ca86-41d7-8287-1af6e94403b4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating instance_info_cache with network_info: [{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Releasing lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Acquiring lock "82a9ec77-951b-4394-b555-00a49710acb5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] Lock "82a9ec77-951b-4394-b555-00a49710acb5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] No waiting events found dispatching network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:10 user nova-compute[71323]: WARNING nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received unexpected event network-vif-plugged-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 for instance with vm_state deleted and task_state None.
Apr 14 03:00:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-67dfc6f9-b5e5-46c9-9b78-cbdc89b8245d req-dfa49f5d-15bd-4c95-9c5c-c2758f9647aa service nova] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Received event network-vif-deleted-35c25a8a-1450-4b7a-8a1f-11c047bb9d27 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG nova.objects.instance [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'flavor' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.048s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:11 user nova-compute[71323]: INFO nova.compute.manager [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Attaching volume c85af65b-ba3d-4e6f-9d4b-0da7c73c091b to /dev/vdb
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.utils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a9cf247-b7a9-4727-b5d5-542bc544fccf]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d518c180-c650-4390-9b55-b64f705f34bf]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[88d4cef0-4582-4052-8cb4-ee59b222e8fe]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d96343b-93d7-4cb2-b989-4ccf32959b42]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:00:11 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:00:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[509dc2b8-4686-4e32-a6b1-d17d078d2250]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG os_brick.utils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] <== get_connector_properties: return (107ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating existing volume attachment record: 1112ebb6-7d87-4efa-97ba-37d8b23357cd {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:00:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:14 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441199.0518413, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:00:14 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Stopped (Lifecycle Event)
Apr 14 03:00:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a4d601aa-955d-49e3-b782-9a37e2463796 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:00:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a4d601aa-955d-49e3-b782-9a37e2463796 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXUbzEVK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:15 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:15 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:15 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[508e3f2c-6592-4c11-bb15-2c26a1b82f18]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:00:15 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9935724-a79e-42a6-9ff7-6596f0c52cb3]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b] added
Apr 14 03:00:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d11de007-a8c5-4b0b-9b34-646f381e4d30]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[59aa5691-af56-4f43-b7a1-b1810e5b981a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kXUbzEVK {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kXUbzEVK" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a88fe23-b339-4463-83bf-4c1731e4b526]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kXUbzEVK'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c11e6f84-4851-4262-888b-da9f011b7a6d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[244db398-0b26-4683-9102-ac95d2207979]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:15 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:15 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[37139c3f-b37c-4f0e-9a3d-cc98a84b2e1f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260]
Apr 14 03:00:15 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2d04ed3-f9cb-4366-aa60-641ea9168c0c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[57e275ff-fabc-4aee-8909-5738ea07a17a]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:16 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:16 user nova-compute[71323]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)
Apr 14 03:00:16 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:16 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:16 user nova-compute[71323]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)
Apr 14 03:00:16 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.274s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] HCTL ('35', '-', '-', 0) found on session 5 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9578a41c-0f9d-4a65-8435-3f120b6c9648]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Searching for a device in session 5 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXUbzEVK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:00:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.305s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] <== connect_volume: return (1306ms) {'type': 'block', 'scsi_wwn': '26265623730383336', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26265623730383336', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.objects.instance [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'flavor' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:00:17 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:17 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:17 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:00:17 user nova-compute[71323]:   <serial>c85af65b-ba3d-4e6f-9d4b-0da7c73c091b</serial>
Apr 14 03:00:17 user nova-compute[71323]: </disk>
Apr 14 03:00:17 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-changed-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Refreshing instance network info cache due to event network-changed-612e6a83-4a68-488b-85d3-c5144d53022c. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] Acquiring lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] Acquired lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Refreshing network info cache for port 612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No VIF found with MAC fa:16:3e:7a:2f:92, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f460826f-ae62-441b-befc-f957a88addab tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.466s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updated VIF entry in instance network info cache for port 612e6a83-4a68-488b-85d3-c5144d53022c. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating instance_info_cache with network_info: [{"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.202", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-593678eb-a744-4351-9a54-a22b9461d2b9 req-32f72a2e-50e3-4d6a-976a-996031c30336 service nova] Releasing lock "refresh_cache-e712dfa7-67a4-4459-9446-94fd8785b09a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:00:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:19 user nova-compute[71323]: INFO nova.compute.manager [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Detaching volume c85af65b-ba3d-4e6f-9d4b-0da7c73c091b
Apr 14 03:00:19 user nova-compute[71323]: INFO nova.virt.block_device [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Attempting to driver detach volume c85af65b-ba3d-4e6f-9d4b-0da7c73c091b from mountpoint /dev/vdb
Apr 14 03:00:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Attempting to detach device vdb from instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:00:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:00:19 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:19 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:19 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:00:19 user nova-compute[71323]:   <serial>c85af65b-ba3d-4e6f-9d4b-0da7c73c091b</serial>
Apr 14 03:00:19 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:00:19 user nova-compute[71323]: </disk>
Apr 14 03:00:19 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:00:19 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully detached device vdb from instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 from the persistent domain config.
Apr 14 03:00:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:00:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:00:19 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:19 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:19 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:00:19 user nova-compute[71323]:   <serial>c85af65b-ba3d-4e6f-9d4b-0da7c73c091b</serial>
Apr 14 03:00:19 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:00:19 user nova-compute[71323]: </disk>
Apr 14 03:00:19 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.objects.instance [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'flavor' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441220.0578115, 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:00:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully detached device vdb from instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 from the live domain config.
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXUbzEVK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:20 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e9f8aab-bc2a-42eb-ba4b-01a79d6fcc72]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:00:20 user nova-compute[71323]: No targets found.
Apr 14 03:00:20 user nova-compute[71323]: iSNS:
Apr 14 03:00:20 user nova-compute[71323]: No targets found.
Apr 14 03:00:20 user nova-compute[71323]: STATIC:
Apr 14 03:00:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf
Apr 14 03:00:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:00:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b
Apr 14 03:00:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:20 user nova-compute[71323]: FIRMWARE:
Apr 14 03:00:20 user nova-compute[71323]: No targets found.
Apr 14 03:00:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:00:20 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:00:20 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.068s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[916f004e-620d-401f-8af5-fa69c42c5ead]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[82e0a40c-c692-41cd-9858-8a998495b76b]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:20 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:20 user nova-compute[71323]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)
Apr 14 03:00:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:20 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:20 user nova-compute[71323]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b (non-flash)
Apr 14 03:00:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b0039bf40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d76040fc-b6f8-46f8-a851-91a4159d9f43]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[06d652a8-644a-4dc2-946d-7e9a59e91332]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7ad23f4-253b-4281-876a-a515619cdc0e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:20 user nova-compute[71323]: INFO nova.compute.manager [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Attaching volume e898aaea-41c4-4fc9-ac07-5d27f62094c4 to /dev/sdc
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.047s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ec86f63-2267-47ed-a5cb-dabdcdba21c2]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[76e9072e-18cb-45c8-a3de-627e11c27eb3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[09dcdf9c-ce7a-4efc-8be5-7257b9cc9e6c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[23194fa3-700a-44e8-a080-df480c75e60d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[74e030fe-1783-4a94-b41f-83e5b1aabfe0]: (4, ('Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260]\nLogout of [sid: 5, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260]
Apr 14 03:00:20 user nova-compute[71323]: Logout of [sid: 5, target: iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d710b75a-9eed-407f-8487-fb8f0ad32be4]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[acddbb13-6dd9-4d61-a528-9be4f4b6bd1e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c85af65b-ba3d-4e6f-9d4b-0da7c73c091b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a9f20e5-4e99-479f-9ce7-780ae45f30e1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.340s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] <== disconnect_volume: return (341ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:00:20 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:00:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef48874f-e43c-44c9-be2d-1af591de77b5]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:20 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating existing volume attachment record: 53380ad5-ca10-49bc-9601-140192017fdf {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:00:21 user nova-compute[71323]: DEBUG nova.objects.instance [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'flavor' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d14f329d-5bc4-4988-9337-40e04e52fb4d tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.736s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:21 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441206.507472, 82a9ec77-951b-4394-b555-00a49710acb5 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:00:21 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] VM Stopped (Lifecycle Event)
Apr 14 03:00:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f5e6ea0e-f21f-4a93-a9af-1586d445f6a1 None None] [instance: 82a9ec77-951b-4394-b555-00a49710acb5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:00:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vmqUDaix', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:24 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[352ba600-57bf-4ab0-a2e9-25ef8346f95f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:00:24 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[99bf23f4-30a0-408c-b83d-b8126b6fe919]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4] added
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b22d066-21fa-475e-b056-0bb6f3422319]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6933686f-8eef-4932-bd8b-4d3081939a6b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vmqUDaix {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vmqUDaix" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2907a1c1-fb16-4091-9baa-05654d1ad917]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'vmqUDaix'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebd0535b-e384-4e8d-84c7-081b7e43a083]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cc51241-5022-454c-915b-a815ea1587c4]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --login" returned: 0 in 0.055s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a8799cf-f03a-41fe-b7c0-84ad894949b1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260]
Apr 14 03:00:24 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c967c9a4-591b-4e78-90ef-f508066e2f36]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[53d5998d-6072-4786-9fe0-1d7d1abd2889]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)
Apr 14 03:00:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.262s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] HCTL ('35', '-', '-', 0) found on session 6 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[af72014e-b003-4a21-be99-bc4c77ea9ab6]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Searching for a device in session 6 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:00:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vmqUDaix', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:00:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.294s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] <== connect_volume: return (1295ms) {'type': 'block', 'scsi_wwn': '26133393363393966', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26133393363393966', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:00:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'flavor' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:00:26 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:26 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:26 user nova-compute[71323]:   <target dev="sdc" bus="scsi"/>
Apr 14 03:00:26 user nova-compute[71323]:   <serial>e898aaea-41c4-4fc9-ac07-5d27f62094c4</serial>
Apr 14 03:00:26 user nova-compute[71323]:   <address type="drive" controller="0" unit="2"/>
Apr 14 03:00:26 user nova-compute[71323]: </disk>
Apr 14 03:00:26 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No BDM found with device name sda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No BDM found with device name sdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No BDM found with device name sdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] No VIF found with MAC fa:16:3e:fe:d0:2d, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-375094ad-6d2b-4cc0-8bbc-06d31f110bc0 tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.374s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:27 user nova-compute[71323]: INFO nova.compute.manager [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Detaching volume e898aaea-41c4-4fc9-ac07-5d27f62094c4
Apr 14 03:00:27 user nova-compute[71323]: INFO nova.virt.block_device [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Attempting to driver detach volume e898aaea-41c4-4fc9-ac07-5d27f62094c4 from mountpoint /dev/sdc
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Attempting to detach device sdc from instance e712dfa7-67a4-4459-9446-94fd8785b09a from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:00:27 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:27 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:27 user nova-compute[71323]:   <target dev="sdc" bus="scsi"/>
Apr 14 03:00:27 user nova-compute[71323]:   <serial>e898aaea-41c4-4fc9-ac07-5d27f62094c4</serial>
Apr 14 03:00:27 user nova-compute[71323]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 14 03:00:27 user nova-compute[71323]: </disk>
Apr 14 03:00:27 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:00:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Successfully detached device sdc from instance e712dfa7-67a4-4459-9446-94fd8785b09a from the persistent domain config.
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] (1/8): Attempting to detach device sdc with device alias scsi0-0-0-2 from instance e712dfa7-67a4-4459-9446-94fd8785b09a from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:00:27 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:00:27 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:00:27 user nova-compute[71323]:   <target dev="sdc" bus="scsi"/>
Apr 14 03:00:27 user nova-compute[71323]:   <serial>e898aaea-41c4-4fc9-ac07-5d27f62094c4</serial>
Apr 14 03:00:27 user nova-compute[71323]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Apr 14 03:00:27 user nova-compute[71323]: </disk>
Apr 14 03:00:27 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441227.6542115, e712dfa7-67a4-4459-9446-94fd8785b09a => scsi0-0-0-2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Start waiting for the detach event from libvirt for device sdc with device alias scsi0-0-0-2 for instance e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:00:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Successfully detached device sdc from instance e712dfa7-67a4-4459-9446-94fd8785b09a from the live domain config.
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vmqUDaix', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:00:27 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cd6f3dd-73ed-496b-9941-cf2c04c33b3c]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:00:27 user nova-compute[71323]: No targets found.
Apr 14 03:00:27 user nova-compute[71323]: iSNS:
Apr 14 03:00:27 user nova-compute[71323]: No targets found.
Apr 14 03:00:27 user nova-compute[71323]: STATIC:
Apr 14 03:00:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf
Apr 14 03:00:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4
Apr 14 03:00:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:00:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:00:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:00:27 user nova-compute[71323]: FIRMWARE:
Apr 14 03:00:27 user nova-compute[71323]: No targets found.
Apr 14 03:00:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:00:27 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:00:27 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b2ccbd-3fb3-48fb-b272-10cd7125e3f4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f299abb5-d3bb-45c5-8b20-54514e3d1be4]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:27 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:27 user nova-compute[71323]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)
Apr 14 03:00:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:00:27 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:00:27 user nova-compute[71323]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 (non-flash)
Apr 14 03:00:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b003b1240>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[26b03a2d-f5e5-4528-9db0-c179dfebf533]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[87154809-f3ef-4d30-87ef-fa8d270973b7]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4931f636-194c-4df1-a93a-4f0e98da32a7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c091d21-affe-4019-af54-d22f13600b7b]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[771d2694-5d4b-4e15-b1db-46f1f02c030d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[56156768-6221-49bc-9bd1-d8199a246ef9]: (4, ('Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260]\nLogout of [sid: 6, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260]
Apr 14 03:00:27 user nova-compute[71323]: Logout of [sid: 6, target: iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4, portal: 172.16.0.220,3260] successful.
Apr 14 03:00:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e898aaea-41c4-4fc9-ac07-5d27f62094c4 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[81ca1139-73ec-4d41-b046-d49b6f9505c4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.284s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] <== disconnect_volume: return (285ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:00:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:00:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:28 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'flavor' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6fe4b60-0ecd-4422-9ede-418ee997e31c tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.623s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:29 user nova-compute[71323]: INFO nova.compute.manager [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Terminating instance
Apr 14 03:00:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-unplugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] No waiting events found dispatching network-vif-unplugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4247b67-2235-4cd0-bb30-4c89e21d2515 req-33f27eaf-d98d-4f04-87ba-820624a2f26c service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-unplugged-612e6a83-4a68-488b-85d3-c5144d53022c for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:00:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Instance destroyed successfully.
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lazy-loading 'resources' on Instance uuid e712dfa7-67a4-4459-9446-94fd8785b09a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T02:58:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-860564790',display_name='tempest-AttachSCSIVolumeTestJSON-server-860564790',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-860564790',id=8,image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDssy5+6+IRBX1/1QfIFiEyGurxnrzN7WU7XXH4VDLLVpl+RHIRY6CDaehyIbJeSPxvckSMHVq4L1xERiuNNi3d0vUlx65kRNHjlURIglNALBrmkJXBAu0neUdjagAUNcA==',key_name='tempest-keypair-1011448847',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bcbeb4e2fdc1420c98c264aac62340a7',ramdisk_id='',reservation_id='r-n6uvqa8r',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='74dfe7d1-36bb-426b-acac-1fe6efabb7b9',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_scsi_model='virtio-scsi',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-AttachSCSIVolumeTestJSON-1798094222',owner_user_name='tempest-AttachSCSIVolumeTestJSON-1798094222-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:58:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='9eabf8254cde4d86a1862934ec9b5240',uuid=e712dfa7-67a4-4459-9446-94fd8785b09a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.202", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converting VIF {"id": "612e6a83-4a68-488b-85d3-c5144d53022c", "address": "fa:16:3e:fe:d0:2d", "network": {"id": "1d9a131e-6bb9-4d3d-a203-b988587252f3", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-860028053-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.202", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bcbeb4e2fdc1420c98c264aac62340a7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap612e6a83-4a", "ovs_interfaceid": "612e6a83-4a68-488b-85d3-c5144d53022c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG os_vif [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap612e6a83-4a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:30 user nova-compute[71323]: INFO os_vif [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fe:d0:2d,bridge_name='br-int',has_traffic_filtering=True,id=612e6a83-4a68-488b-85d3-c5144d53022c,network=Network(1d9a131e-6bb9-4d3d-a203-b988587252f3),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap612e6a83-4a')
Apr 14 03:00:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Deleting instance files /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a_del
Apr 14 03:00:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Deletion of /opt/stack/data/nova/instances/e712dfa7-67a4-4459-9446-94fd8785b09a_del complete
Apr 14 03:00:30 user nova-compute[71323]: INFO nova.compute.manager [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Took 1.14 seconds to destroy the instance on the hypervisor.
Apr 14 03:00:30 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:00:30 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:00:31 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:31 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Took 0.88 seconds to deallocate network for instance.
Apr 14 03:00:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-9adcf87e-ab63-4fe3-a723-6c4f3fb1e5c8 req-efec54dc-b0a4-43f5-a785-38f155e57455 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-deleted-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:31 user nova-compute[71323]: INFO nova.compute.manager [req-9adcf87e-ab63-4fe3-a723-6c4f3fb1e5c8 req-efec54dc-b0a4-43f5-a785-38f155e57455 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Neutron deleted interface 612e6a83-4a68-488b-85d3-c5144d53022c; detaching it from the instance and deleting it from the info cache
Apr 14 03:00:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-9adcf87e-ab63-4fe3-a723-6c4f3fb1e5c8 req-efec54dc-b0a4-43f5-a785-38f155e57455 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-9adcf87e-ab63-4fe3-a723-6c4f3fb1e5c8 req-efec54dc-b0a4-43f5-a785-38f155e57455 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Detach interface failed, port_id=612e6a83-4a68-488b-85d3-c5144d53022c, reason: Instance e712dfa7-67a4-4459-9446-94fd8785b09a could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:00:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] Acquiring lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] No waiting events found dispatching network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:00:32 user nova-compute[71323]: WARNING nova.compute.manager [req-b085757f-fa83-44fb-aa54-56cd74bca8f5 req-2e9d3624-4c67-4825-ac14-f9456c019b87 service nova] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Received unexpected event network-vif-plugged-612e6a83-4a68-488b-85d3-c5144d53022c for instance with vm_state deleted and task_state None.
Apr 14 03:00:32 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:00:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.400s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:32 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Deleted allocations for instance e712dfa7-67a4-4459-9446-94fd8785b09a
Apr 14 03:00:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf02401d-4fb7-4ec3-b8fc-53277e38baeb tempest-AttachSCSIVolumeTestJSON-1798094222 tempest-AttachSCSIVolumeTestJSON-1798094222-project-member] Lock "e712dfa7-67a4-4459-9446-94fd8785b09a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.645s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:45 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441230.4132307, e712dfa7-67a4-4459-9446-94fd8785b09a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:00:45 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] VM Stopped (Lifecycle Event)
Apr 14 03:00:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d86f839e-97cf-43f1-95c2-7713b0f443c2 None None] [instance: e712dfa7-67a4-4459-9446-94fd8785b09a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:00:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:00:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:00:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:00:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:00:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.126s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:00:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.126s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:00:54 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:00:54 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:00:54 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8044MB free_disk=26.665138244628906GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:00:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc99f782-81c1-4243-8853-5554e5663d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a826438-d96e-4e70-b50e-aca4be008394 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.428s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:00:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:00:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:01:10 user nova-compute[71323]: INFO nova.compute.claims [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Claim successful on node user
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.383s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:01:11 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.policy [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f0d4916eb93c4b3ebbbd2af4b0cdacc1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94a1eb61159b43228db0d529a5be816d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:01:11 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Creating image(s)
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "/opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "/opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "/opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk 1073741824" returned: 0 in 0.046s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.183s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Checking if we can resize image /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Cannot resize image /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'migration_context' on Instance uuid 287a236d-d191-465f-bb65-636732ce5689 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Ensure instance console log exists: /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Successfully created port: 7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Successfully updated port: 7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquired lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-changed-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Refreshing instance network info cache due to event network-changed-7a4cd76a-7605-4209-9dd2-ec4bced26f20. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] Acquiring lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Updating instance_info_cache with network_info: [{"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Releasing lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Instance network_info: |[{"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] Acquired lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Refreshing network info cache for port 7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Start _get_guest_xml network_info=[{"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:01:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-437824819',display_name='tempest-VolumesAdminNegativeTest-server-437824819',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-437824819',id=12,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-cwcb1gb7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:11Z,user_data=None,user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=287a236d-d191-465f-bb65-636732ce5689,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.objects.instance [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'pci_devices' on Instance uuid 287a236d-d191-465f-bb65-636732ce5689 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:01:13 user nova-compute[71323]:   <uuid>287a236d-d191-465f-bb65-636732ce5689</uuid>
Apr 14 03:01:13 user nova-compute[71323]:   <name>instance-0000000c</name>
Apr 14 03:01:13 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:01:13 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:01:13 user nova-compute[71323]:   <metadata>
Apr 14 03:01:13 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:01:13 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:name>tempest-VolumesAdminNegativeTest-server-437824819</nova:name>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:01:13</nova:creationTime>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:01:13 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:01:13 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:owner>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:user uuid="f0d4916eb93c4b3ebbbd2af4b0cdacc1">tempest-VolumesAdminNegativeTest-1232189050-project-member</nova:user>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:project uuid="94a1eb61159b43228db0d529a5be816d">tempest-VolumesAdminNegativeTest-1232189050</nova:project>
Apr 14 03:01:13 user nova-compute[71323]:       </nova:owner>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:01:13 user nova-compute[71323]:       <nova:ports>
Apr 14 03:01:13 user nova-compute[71323]:         <nova:port uuid="7a4cd76a-7605-4209-9dd2-ec4bced26f20">
Apr 14 03:01:13 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:01:13 user nova-compute[71323]:         </nova:port>
Apr 14 03:01:13 user nova-compute[71323]:       </nova:ports>
Apr 14 03:01:13 user nova-compute[71323]:     </nova:instance>
Apr 14 03:01:13 user nova-compute[71323]:   </metadata>
Apr 14 03:01:13 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:01:13 user nova-compute[71323]:     <system>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="serial">287a236d-d191-465f-bb65-636732ce5689</entry>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="uuid">287a236d-d191-465f-bb65-636732ce5689</entry>
Apr 14 03:01:13 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:01:13 user nova-compute[71323]:     </system>
Apr 14 03:01:13 user nova-compute[71323]:   </sysinfo>
Apr 14 03:01:13 user nova-compute[71323]:   <os>
Apr 14 03:01:13 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:01:13 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:01:13 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:01:13 user nova-compute[71323]:   </os>
Apr 14 03:01:13 user nova-compute[71323]:   <features>
Apr 14 03:01:13 user nova-compute[71323]:     <acpi/>
Apr 14 03:01:13 user nova-compute[71323]:     <apic/>
Apr 14 03:01:13 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:01:13 user nova-compute[71323]:   </features>
Apr 14 03:01:13 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:01:13 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:01:13 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:01:13 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:01:13 user nova-compute[71323]:   </clock>
Apr 14 03:01:13 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:01:13 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:01:13 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:01:13 user nova-compute[71323]:   </cpu>
Apr 14 03:01:13 user nova-compute[71323]:   <devices>
Apr 14 03:01:13 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:01:13 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:01:13 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk"/>
Apr 14 03:01:13 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:01:13 user nova-compute[71323]:     </disk>
Apr 14 03:01:13 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:01:13 user nova-compute[71323]:       <mac address="fa:16:3e:6b:95:3f"/>
Apr 14 03:01:13 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:01:13 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:01:13 user nova-compute[71323]:       <target dev="tap7a4cd76a-76"/>
Apr 14 03:01:13 user nova-compute[71323]:     </interface>
Apr 14 03:01:13 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:01:13 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/console.log" append="off"/>
Apr 14 03:01:13 user nova-compute[71323]:     </serial>
Apr 14 03:01:13 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:01:13 user nova-compute[71323]:     <video>
Apr 14 03:01:13 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:01:13 user nova-compute[71323]:     </video>
Apr 14 03:01:13 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:01:13 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:01:13 user nova-compute[71323]:     </rng>
Apr 14 03:01:13 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:01:13 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:01:13 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:01:13 user nova-compute[71323]:     </memballoon>
Apr 14 03:01:13 user nova-compute[71323]:   </devices>
Apr 14 03:01:13 user nova-compute[71323]: </domain>
Apr 14 03:01:13 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-437824819',display_name='tempest-VolumesAdminNegativeTest-server-437824819',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-437824819',id=12,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-cwcb1gb7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:11Z,user_data=None,user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=287a236d-d191-465f-bb65-636732ce5689,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG os_vif [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7a4cd76a-76, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7a4cd76a-76, col_values=(('external_ids', {'iface-id': '7a4cd76a-7605-4209-9dd2-ec4bced26f20', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6b:95:3f', 'vm-uuid': '287a236d-d191-465f-bb65-636732ce5689'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:13 user nova-compute[71323]: INFO os_vif [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76')
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] No VIF found with MAC fa:16:3e:6b:95:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Updated VIF entry in instance network info cache for port 7a4cd76a-7605-4209-9dd2-ec4bced26f20. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Updating instance_info_cache with network_info: [{"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-631add27-4af4-4df4-bacc-ee96f2904807 req-7c063446-8569-469d-98b8-e0187f4fab06 service nova] Releasing lock "refresh_cache-287a236d-d191-465f-bb65-636732ce5689" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] No waiting events found dispatching network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:15 user nova-compute[71323]: WARNING nova.compute.manager [req-5c25b03d-649d-420e-9f1a-b70d9ecd6e3b req-57f59666-d460-4bfd-aefe-802c91f6e657 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received unexpected event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 for instance with vm_state building and task_state spawning.
Apr 14 03:01:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441277.0063348, 287a236d-d191-465f-bb65-636732ce5689 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] VM Resumed (Lifecycle Event)
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] Instance spawned successfully.
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441277.0080113, 287a236d-d191-465f-bb65-636732ce5689 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] VM Started (Lifecycle Event)
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] No waiting events found dispatching network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:17 user nova-compute[71323]: WARNING nova.compute.manager [req-46f67978-e26d-4406-a397-d132617d56a1 req-f0a4c4bb-6453-45b5-83d9-6085b279bea7 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received unexpected event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 for instance with vm_state building and task_state spawning.
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Took 5.75 seconds to spawn the instance on the hypervisor.
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:17 user nova-compute[71323]: INFO nova.compute.manager [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Took 6.43 seconds to build instance.
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94987643-382c-41c4-8a92-acc3156dd69f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.523s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing instance network info cache due to event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] Acquiring lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] Acquired lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG nova.objects.instance [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.058s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG nova.network.neutron [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updated VIF entry in instance network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG nova.network.neutron [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating instance_info_cache with network_info: [{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.246", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cf2f12d6-2e84-45d6-a50b-772ae60d6461 req-294eb708-4164-4148-ba38-248626f8c9e4 service nova] Releasing lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:18 user nova-compute[71323]: INFO nova.compute.manager [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Attaching volume edd92bd4-26a2-4652-bdc6-77805711142a to /dev/vdb
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.utils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[488c4e1b-77a5-42d7-818d-5b8c12831645]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d71a308-e037-4a92-aae9-a12b4ad3aabb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[da729f5c-8e50-491e-9352-8c5742eaff60]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[714ea2f6-ae6f-4167-b6ba-1cb04c8aced4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:18 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:18 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fff328be-c39d-476a-943a-bf4bd391a60d]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG os_brick.utils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating existing volume attachment record: a532a8a2-f086-40cb-8ce1-d9a8b50a753f {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXtdrAGB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:23 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:23 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:23 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260" returned: 21 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28dc9264-7447-4f8c-a38d-08a37fd4c33d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:01:23 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93677c68-b630-41bb-90d9-8f9127d38fc1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a] added
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[250d7886-4bc6-4f34-9003-bb1a63653185]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[093fd6fd-527f-455a-8a17-f22003cb51d5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kXtdrAGB {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kXtdrAGB" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c88eb0d-65dc-4521-b5b3-ab4a9a0c5b0e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kXtdrAGB'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[042846a8-5628-4544-a2b1-9387f5e51845]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de894122-bd67-485d-974f-975709885e99]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[924f5571-be8a-4044-bb23-9ec54a7eb5d8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260]
Apr 14 03:01:23 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[98fe01d8-afa2-4139-8627-b9abda26a2a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e77d1736-32bd-40c0-a6d8-6716993677f0]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:23 user nova-compute[71323]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)
Apr 14 03:01:23 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.331s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] HCTL ('35', '-', '-', 0) found on session 7 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cb7b454-829e-41bc-ad3a-4c363d8a85a4]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Searching for a device in session 7 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:01:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXtdrAGB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.370s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== connect_volume: return (1372ms) {'type': 'block', 'scsi_wwn': '23062393431376364', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062393431376364', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.objects.instance [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:01:24 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:24 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:01:24 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:01:24 user nova-compute[71323]:   <serial>edd92bd4-26a2-4652-bdc6-77805711142a</serial>
Apr 14 03:01:24 user nova-compute[71323]: </disk>
Apr 14 03:01:24 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:01:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No VIF found with MAC fa:16:3e:93:69:3c, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:01:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-50a8fbda-d9b1-4e0e-b933-627604850134 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.901s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:27 user nova-compute[71323]: INFO nova.compute.manager [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Detaching volume edd92bd4-26a2-4652-bdc6-77805711142a
Apr 14 03:01:27 user nova-compute[71323]: INFO nova.virt.block_device [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Attempting to driver detach volume edd92bd4-26a2-4652-bdc6-77805711142a from mountpoint /dev/vdb
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Attempting to detach device vdb from instance 2ca84990-b165-4d0b-a917-307cd8f96f28 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:01:27 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:27 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:01:27 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:01:27 user nova-compute[71323]:   <serial>edd92bd4-26a2-4652-bdc6-77805711142a</serial>
Apr 14 03:01:27 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:01:27 user nova-compute[71323]: </disk>
Apr 14 03:01:27 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:01:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully detached device vdb from instance 2ca84990-b165-4d0b-a917-307cd8f96f28 from the persistent domain config.
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 2ca84990-b165-4d0b-a917-307cd8f96f28 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:01:27 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:27 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:01:27 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:01:27 user nova-compute[71323]:   <serial>edd92bd4-26a2-4652-bdc6-77805711142a</serial>
Apr 14 03:01:27 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:01:27 user nova-compute[71323]: </disk>
Apr 14 03:01:27 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:27 user nova-compute[71323]: INFO nova.compute.manager [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Terminating instance
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441287.904011, 2ca84990-b165-4d0b-a917-307cd8f96f28 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:01:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully detached device vdb from instance 2ca84990-b165-4d0b-a917-307cd8f96f28 from the live domain config.
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kXtdrAGB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:27 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-unplugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] No waiting events found dispatching network-vif-unplugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-47c24f6b-439c-4a22-bfc7-5ce6462626d6 req-7e500806-2b5d-49d2-998d-30fc39f72399 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-unplugged-c75de928-a52a-4a2a-b413-15a2be115846 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[db80ca2c-407a-476d-8f31-8adfb2ef2728]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:01:27 user nova-compute[71323]: No targets found.
Apr 14 03:01:27 user nova-compute[71323]: iSNS:
Apr 14 03:01:27 user nova-compute[71323]: No targets found.
Apr 14 03:01:27 user nova-compute[71323]: STATIC:
Apr 14 03:01:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf
Apr 14 03:01:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:01:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:27 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a
Apr 14 03:01:27 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:27 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:27 user nova-compute[71323]: FIRMWARE:
Apr 14 03:01:27 user nova-compute[71323]: No targets found.
Apr 14 03:01:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:01:27 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:01:27 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbfad40e-a859-405f-8d95-f377c080c4c4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcec1e88-2e80-4408-9dc7-bced9990a75d]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a (non-flash)
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b0039bac0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[847c6cad-d9d9-4c5d-9285-438330fe2dca]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[096020cf-70c4-4804-844f-21a501f9a8aa]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ac81416-9104-4b14-a1a3-49c8b799667b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.053s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c8b320c-9737-4080-b8c5-63e06c40969a]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[54c36487-affa-401c-a253-9c565ef84c1b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ae6e704-df42-40b7-beb7-2d98e7f9c0f0]: (4, ('Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260]\nLogout of [sid: 7, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260]
Apr 14 03:01:28 user nova-compute[71323]: Logout of [sid: 7, target: iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:edd92bd4-26a2-4652-bdc6-77805711142a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c79b63ce-44f1-420e-991e-c919c289d730]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.372s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== disconnect_volume: return (374ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Instance destroyed successfully.
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.objects.instance [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid bc99f782-81c1-4243-8853-5554e5663d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:36Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-432581934',display_name='tempest-TestVolumeBootPattern-server-432581934',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-432581934',id=10,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:59:56Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-fnw79iau',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:59:57Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=bc99f782-81c1-4243-8853-5554e5663d0a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "c75de928-a52a-4a2a-b413-15a2be115846", "address": "fa:16:3e:79:f5:42", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc75de928-a5", "ovs_interfaceid": "c75de928-a52a-4a2a-b413-15a2be115846", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_vif [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc75de928-a5, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:28 user nova-compute[71323]: INFO os_vif [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:f5:42,bridge_name='br-int',has_traffic_filtering=True,id=c75de928-a52a-4a2a-b413-15a2be115846,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc75de928-a5')
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JcvzNKnb', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:28 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3573aff3-911a-4061-a51e-95274826c950]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:01:28 user nova-compute[71323]: No targets found.
Apr 14 03:01:28 user nova-compute[71323]: iSNS:
Apr 14 03:01:28 user nova-compute[71323]: No targets found.
Apr 14 03:01:28 user nova-compute[71323]: STATIC:
Apr 14 03:01:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf
Apr 14 03:01:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:01:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:28 user nova-compute[71323]: FIRMWARE:
Apr 14 03:01:28 user nova-compute[71323]: No targets found.
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:01:28 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:01:28 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce0abd61-dc1a-4458-aa1e-8d9d9c043807]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[29441b23-af0e-483b-a7c1-37448d7db059]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf (non-flash)
Apr 14 03:01:28 user nova-compute[71323]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd57e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[56117651-c088-4c75-8d14-fd20b2be2be3]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[795fe222-e0e3-423a-b212-2b37b94bc5b4]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[16617edf-45b8-4464-ab8d-21d08f0291a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a6ec0d6-0a17-43aa-b735-ec99ecfa50ec]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23861323933633364', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_eb79ce07', '/dev/disk/by-id/scsi-1SCST_BIOeb79ce07-8a293c3d4c1dbbfe') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:01:28 user nova-compute[71323]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-23861323933633364
Apr 14 03:01:28 user nova-compute[71323]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-SSCST_BIO_Storage_eb79ce07
Apr 14 03:01:28 user nova-compute[71323]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIOeb79ce07-8a293c3d4c1dbbfe
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28ea0367-94e3-44c8-83f9-916eb474eb34]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[206fae79-b631-44e2-9d7e-3caca5f549d3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21289c9e-f1e1-4db8-b6bc-469263b8ab7c]: (4, ('Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260]\nLogout of [sid: 2, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260]
Apr 14 03:01:28 user nova-compute[71323]: Logout of [sid: 2, target: iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:47e890f1-6257-486b-ab68-7d91ac9f65bf -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e43b50a-8231-4b5d-aa7b-e22a94f5cefe]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (364ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:01:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Deleting instance files /opt/stack/data/nova/instances/bc99f782-81c1-4243-8853-5554e5663d0a_del
Apr 14 03:01:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Deletion of /opt/stack/data/nova/instances/bc99f782-81c1-4243-8853-5554e5663d0a_del complete
Apr 14 03:01:28 user nova-compute[71323]: INFO nova.compute.manager [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Took 1.31 seconds to destroy the instance on the hypervisor.
Apr 14 03:01:28 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:01:28 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:01:29 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f2f9345-9ca2-4a47-bcaf-9f3287ed6544 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.996s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] Acquiring lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] Lock "bc99f782-81c1-4243-8853-5554e5663d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] No waiting events found dispatching network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:30 user nova-compute[71323]: WARNING nova.compute.manager [req-2a19afd9-722d-4fc4-b7b3-52351ef66d76 req-2c45d025-6f08-465e-ae20-c41bc4f8069c service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received unexpected event network-vif-plugged-c75de928-a52a-4a2a-b413-15a2be115846 for instance with vm_state active and task_state deleting.
Apr 14 03:01:30 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:30 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Took 1.71 seconds to deallocate network for instance.
Apr 14 03:01:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:31 user nova-compute[71323]: INFO nova.compute.manager [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Terminating instance
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:01:31 user nova-compute[71323]: INFO nova.compute.manager [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Took 0.81 seconds to detach 1 volumes for instance.
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Deleting volume: 47e890f1-6257-486b-ab68-7d91ac9f65bf {{(pid=71323) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Instance destroyed successfully.
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'resources' on Instance uuid 8a826438-d96e-4e70-b50e-aca4be008394 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-408869365',display_name='tempest-ServersNegativeTestJSON-server-408869365',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-408869365',id=11,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:59:47Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-clw8zz05',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:59:47Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=8a826438-d96e-4e70-b50e-aca4be008394,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "8151f818-904f-472f-b66c-ee41b2c2975e", "address": "fa:16:3e:aa:18:3d", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8151f818-90", "ovs_interfaceid": "8151f818-904f-472f-b66c-ee41b2c2975e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG os_vif [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8151f818-90, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:31 user nova-compute[71323]: INFO os_vif [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:18:3d,bridge_name='br-int',has_traffic_filtering=True,id=8151f818-904f-472f-b66c-ee41b2c2975e,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8151f818-90')
Apr 14 03:01:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Deleting instance files /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394_del
Apr 14 03:01:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Deletion of /opt/stack/data/nova/instances/8a826438-d96e-4e70-b50e-aca4be008394_del complete
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.410s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:32 user nova-compute[71323]: INFO nova.compute.manager [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Received event network-vif-deleted-c75de928-a52a-4a2a-b413-15a2be115846 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-unplugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] No waiting events found dispatching network-vif-unplugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a2bb4147-bbf7-4557-8bf1-67279e40ff4f req-85e085e1-66cb-409e-981c-c396a3178cd6 service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-unplugged-8151f818-904f-472f-b66c-ee41b2c2975e for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:01:32 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance bc99f782-81c1-4243-8853-5554e5663d0a
Apr 14 03:01:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f59106a4-de65-4fbb-b377-2adee8c2f4cb tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "bc99f782-81c1-4243-8853-5554e5663d0a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.559s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:33 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Took 1.13 seconds to deallocate network for instance.
Apr 14 03:01:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.383s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:33 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Deleted allocations for instance 8a826438-d96e-4e70-b50e-aca4be008394
Apr 14 03:01:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ec99f6d5-92b2-439d-967a-39607a11686a tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "8a826438-d96e-4e70-b50e-aca4be008394" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.455s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] Acquiring lock "8a826438-d96e-4e70-b50e-aca4be008394-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] Lock "8a826438-d96e-4e70-b50e-aca4be008394-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] No waiting events found dispatching network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:34 user nova-compute[71323]: WARNING nova.compute.manager [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received unexpected event network-vif-plugged-8151f818-904f-472f-b66c-ee41b2c2975e for instance with vm_state deleted and task_state None.
Apr 14 03:01:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-f8f10b28-1483-461b-88a7-5ded9afecd56 req-bbcb7108-1017-409e-94a8-763b77fd871f service nova] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Received event network-vif-deleted-8151f818-904f-472f-b66c-ee41b2c2975e {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441288.3960407, bc99f782-81c1-4243-8853-5554e5663d0a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:43 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] VM Stopped (Lifecycle Event)
Apr 14 03:01:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2abdf874-0c88-4973-a18b-2af26ad707e9 None None] [instance: bc99f782-81c1-4243-8853-5554e5663d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:01:43 user nova-compute[71323]: INFO nova.compute.claims [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Claim successful on node user
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.394s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:01:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:01:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:01:44 user nova-compute[71323]: INFO nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Booting with volume ba0a016e-f7ab-4aac-8fb3-ecf199678570 at /dev/vda
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cea390bc-2165-43ad-b381-765080c38ad9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG nova.policy [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c3c342144e51496190e32329225ddd2b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de2d10e0d7e7477484e4204841a3980c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1793edc7-fef4-44a5-b1ae-b20e12209b96]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecd106c0-4d4a-4afd-9108-23d41766ad3b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[12bc9e61-7940-4c6a-8faa-99eb731d0c6e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:44 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4612c206-e96a-4909-921c-129d2dc432f6]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:44 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating existing volume attachment record: 7bc346aa-0798-431e-9e3c-58701616304e {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully created port: 7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'flavor' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:45 user nova-compute[71323]: INFO nova.compute.manager [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attaching volume f0c626e1-cd60-46d3-b894-a000a5cb3811 to /dev/vdb
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.utils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[705d24a1-08cf-4dfc-a974-74319a220cf4]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b81779b-f25f-40df-b468-d5f88590cb70]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dfbb1ec-aa2d-41d5-a601-dd65a34bece9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fdf8cf6-0879-4b09-9a57-02ba6a0122b1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:45 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a6e3bdf-deae-4c9c-824f-5f599e1c0a35]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG os_brick.utils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating existing volume attachment record: 23f3e358-cbc4-4ae7-935e-d761f2bdeb3f {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully created port: 04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:46 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441291.9445698, 8a826438-d96e-4e70-b50e-aca4be008394 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:46 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] VM Stopped (Lifecycle Event)
Apr 14 03:01:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2aa26acb-a776-4365-b3dc-cb7d0b067629 None None] [instance: 8a826438-d96e-4e70-b50e-aca4be008394] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:47 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully created port: fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully created port: e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'info_cache' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully created port: 65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:01:49 user nova-compute[71323]: INFO nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Booting with volume 453b7855-8efd-41bd-806c-18ae2c03ac49 at /dev/vdb
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa08dbd3-2631-43c1-9702-95682788fc02]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dfd3ac7-d88f-40a8-aa3c-ca6c5d0bc310]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bbe4ad2-720c-485d-b4a1-f914652f2bf6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbae2b6d-acdf-4bc6-83e8-21e25a7a64f5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:49 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f184e29-e267-4cc9-9f57-44a0f96b5a93]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== get_connector_properties: return (131ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating existing volume attachment record: 897df238-c72c-46c7-80f3-6844d072066a {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: 7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-7e8d8948-ff3b-461a-921a-d90f46d9f84c. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance destroyed successfully.
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'resources' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:59:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb85a839f-97, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:49 user nova-compute[71323]: INFO os_vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 03:01:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:49 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1576583-f297-4539-9098-ac45c287aa60]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:01:49 user nova-compute[71323]: No targets found.
Apr 14 03:01:49 user nova-compute[71323]: iSNS:
Apr 14 03:01:49 user nova-compute[71323]: No targets found.
Apr 14 03:01:49 user nova-compute[71323]: STATIC:
Apr 14 03:01:49 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:01:49 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:49 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:49 user nova-compute[71323]: FIRMWARE:
Apr 14 03:01:49 user nova-compute[71323]: No targets found.
Apr 14 03:01:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:01:49 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:01:49 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[110110e7-b46c-4932-baa9-3633603dabfd]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[049298a9-a53d-4895-8f55-389c5c8f5d42]: (4, ('tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd63b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[697a4e6f-9655-4252-a476-579a60cb3e84]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21b2342c-8a60-44b9-91f9-64f79bfa2656]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b04266d-cd9c-4676-a9fb-6b987d28337b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.036s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fe4cd05-c580-417a-9070-b44c7400eeaf]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[59e1fb6b-9818-4e85-8949-189ec248a731]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[41fa1e56-c200-420e-b2e0-6025988fca70]: (4, ('Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]\nLogout of [sid: 3, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]
Apr 14 03:01:50 user nova-compute[71323]: Logout of [sid: 3, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[edf51dc6-bf84-4ec5-974d-ff6ea693d07c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.305s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== disconnect_volume: return (307ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start _get_guest_xml network_info=[{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '74d9f89b-3416-45c5-8af8-870134e07548', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': 'bbb6cd58-ce7a-43ae-876d-51a888193f60', 'attached_at': '', 'detached_at': '', 'volume_id': 'd1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'serial': 'd1ac2d2b-c18f-4958-a86e-fa298a15c9b8'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'numa_topology' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'vcpu_model' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:50 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[16499e33-8392-4693-a86f-f0385289a12b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:01:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a48f0f8d-17df-41b9-9d55-e7c295d4d7b0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8] added
Apr 14 03:01:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8eada9fe-d098-4dbe-b975-c2a3ab4512b7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[305d1166-533f-487d-91a0-05f8190a34d7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mpkibDRE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mpkibDRE" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ff58cb7-f917-40b2-a261-3272726bd896]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mpkibDRE'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c53a250e-b983-47e6-bf31-a4dc7dd4fca2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e3e8e61-27b4-4f51-81e0-29ee2bd49869]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:01:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:01:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 03:01:50 user nova-compute[71323]: 
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --login" returned: 0 in 0.047s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1df7f7bb-cb1d-4665-b3eb-c460c2af5c58]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]
Apr 14 03:01:50 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3427e5e4-1a34-467f-910d-7b359ffe3a95]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: 5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f62ecae9-50a0-4783-838d-eb3789722c21 req-83fd9ae8-e191-42cc-8255-100ebb1e7039 service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a1d80e3-bbf1-4bdb-8ca0-b3c30c94ca9e]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.268s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] HCTL ('33', '-', '-', 0) found on session 8 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[434b87ad-6e1e-4655-b093-2b01ca93e1de]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Searching for a device in session 8 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:01:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: 3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-3e1675a7-2e92-4b45-a657-f7b811975097. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.300s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== connect_volume: return (1302ms) {'type': 'block', 'scsi_wwn': '23531323766386562', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23531323766386562', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.264s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:51 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:51 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[11011757-731a-4c0a-a709-7f5280b5a8b3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:01:51 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:59:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'pci_devices' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:01:51 user nova-compute[71323]:   <uuid>bbb6cd58-ce7a-43ae-876d-51a888193f60</uuid>
Apr 14 03:01:51 user nova-compute[71323]:   <name>instance-00000002</name>
Apr 14 03:01:51 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:01:51 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:01:51 user nova-compute[71323]:   <metadata>
Apr 14 03:01:51 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:01:51 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeTestJSON-server-1854200462</nova:name>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:01:50</nova:creationTime>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:01:51 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:01:51 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:owner>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:user uuid="d6d78d4991b5493e8893034e91ed407b">tempest-AttachVolumeTestJSON-975876676-project-member</nova:user>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:project uuid="130665008cf648cf87467ae2777a9170">tempest-AttachVolumeTestJSON-975876676</nova:project>
Apr 14 03:01:51 user nova-compute[71323]:       </nova:owner>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:01:51 user nova-compute[71323]:       <nova:ports>
Apr 14 03:01:51 user nova-compute[71323]:         <nova:port uuid="b85a839f-973d-415e-94b6-cdb480a8b89d">
Apr 14 03:01:51 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:01:51 user nova-compute[71323]:         </nova:port>
Apr 14 03:01:51 user nova-compute[71323]:       </nova:ports>
Apr 14 03:01:51 user nova-compute[71323]:     </nova:instance>
Apr 14 03:01:51 user nova-compute[71323]:   </metadata>
Apr 14 03:01:51 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:01:51 user nova-compute[71323]:     <system>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="serial">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="uuid">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 03:01:51 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:01:51 user nova-compute[71323]:     </system>
Apr 14 03:01:51 user nova-compute[71323]:   </sysinfo>
Apr 14 03:01:51 user nova-compute[71323]:   <os>
Apr 14 03:01:51 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:01:51 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:01:51 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:01:51 user nova-compute[71323]:   </os>
Apr 14 03:01:51 user nova-compute[71323]:   <features>
Apr 14 03:01:51 user nova-compute[71323]:     <acpi/>
Apr 14 03:01:51 user nova-compute[71323]:     <apic/>
Apr 14 03:01:51 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:01:51 user nova-compute[71323]:   </features>
Apr 14 03:01:51 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:01:51 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:01:51 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:01:51 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:01:51 user nova-compute[71323]:   </clock>
Apr 14 03:01:51 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:01:51 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:01:51 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:01:51 user nova-compute[71323]:   </cpu>
Apr 14 03:01:51 user nova-compute[71323]:   <devices>
Apr 14 03:01:51 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:01:51 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:01:51 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk"/>
Apr 14 03:01:51 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:01:51 user nova-compute[71323]:     </disk>
Apr 14 03:01:51 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:01:51 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:51 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:01:51 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:01:51 user nova-compute[71323]:       <serial>d1ac2d2b-c18f-4958-a86e-fa298a15c9b8</serial>
Apr 14 03:01:51 user nova-compute[71323]:     </disk>
Apr 14 03:01:51 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:01:51 user nova-compute[71323]:       <mac address="fa:16:3e:f7:89:a6"/>
Apr 14 03:01:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:01:51 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:01:51 user nova-compute[71323]:       <target dev="tapb85a839f-97"/>
Apr 14 03:01:51 user nova-compute[71323]:     </interface>
Apr 14 03:01:51 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:01:51 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/console.log" append="off"/>
Apr 14 03:01:51 user nova-compute[71323]:     </serial>
Apr 14 03:01:51 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:01:51 user nova-compute[71323]:     <video>
Apr 14 03:01:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:01:51 user nova-compute[71323]:     </video>
Apr 14 03:01:51 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:01:51 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:01:51 user nova-compute[71323]:     </rng>
Apr 14 03:01:51 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:01:51 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:01:51 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:01:51 user nova-compute[71323]:     </memballoon>
Apr 14 03:01:51 user nova-compute[71323]:   </devices>
Apr 14 03:01:51 user nova-compute[71323]: </domain>
Apr 14 03:01:51 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fda5138a-9f87-486d-b792-5720ead7b4f4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811] added
Apr 14 03:01:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-5b8574e2-f175-4432-a1cb-f17916ddc806. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dad86889-1c8c-48ba-a477-be4fec1242f3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4cd92378-5f9f-4378-82d5-a11742982060]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cpDvVdUM {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cpDvVdUM" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fdc5a05-acc3-4c7f-a564-4c9639fb41e3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cpDvVdUM'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-24ceee17-3070-4c41-a073-4a5be0d51235 req-03e694c8-acff-490c-8755-64abc42d811a service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b53ac7d-68c4-4503-817c-ddf128a77b9d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7dadb8c4-69bc-444d-abab-923f6183974e]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3088e489-af82-44f1-80e3-8634488e95e9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]
Apr 14 03:01:51 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'trusted_certs' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0de40004-61c3-41d5-bbe0-e806f8fc6c8e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[30035928-d4c5-498e-8659-e0803746fa33]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:52 user nova-compute[71323]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:01:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:52 user nova-compute[71323]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:01:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.474s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('34', '-', '-', 0) found on session 9 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[379d77ae-5ec1-47ab-a8fe-adf71db732cd]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 9 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: 04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0b35235-b9db-4c6d-961c-2107227f102b req-7b05347c-d944-4ab3-952e-cc2db524d7cc service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.210s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking if we can resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Cannot resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'migration_context' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:59:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG os_vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb85a839f-97, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb85a839f-97, col_values=(('external_ids', {'iface-id': 'b85a839f-973d-415e-94b6-cdb480a8b89d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:89:a6', 'vm-uuid': 'bbb6cd58-ce7a-43ae-876d-51a888193f60'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:52 user nova-compute[71323]: INFO os_vif [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:01:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.521s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (2785ms) {'type': 'block', 'scsi_wwn': '26336373965373062', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26336373965373062', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'flavor' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:01:53 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:53 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 03:01:53 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:01:53 user nova-compute[71323]:   <serial>f0c626e1-cd60-46d3-b894-a000a5cb3811</serial>
Apr 14 03:01:53 user nova-compute[71323]: </disk>
Apr 14 03:01:53 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-04ec9de5-1982-4641-8e4f-6eed7f00cf51. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:af:bb:dd, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-fa3f5b14-6337-4e66-b990-1ba1cb8ab667. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-29f8a1f8-b1d4-42cd-8c2c-5f0368646eed tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.516s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb864f17-c31d-4b7f-8a2e-b4d5c7367a67 req-7ea53fe4-8ba8-488d-b84c-4a7648c2df4f service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:54 user nova-compute[71323]: INFO nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Booting with volume c11ac81e-755b-4855-85b5-259c815fef2c at /dev/vdc
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f98f166-a1ce-44cf-a8ff-6977cef69f22]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[357c17eb-54bb-4864-80cf-f93561f76335]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[611bfd23-1b67-4275-b5e7-a7fb135fb3dd]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5af21dfd-c861-4ef4-80a9-5d2641f7d953]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:54 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6af2025c-40ce-4689-adea-6ef7476fc641]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG os_brick.utils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== get_connector_properties: return (134ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating existing volume attachment record: d1db064f-367e-4d97-a3dc-7007a652c17b {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:54 user nova-compute[71323]: WARNING nova.compute.manager [req-bb921526-e84f-4b18-a128-3917d3ec30cc req-822f1a10-2948-41d1-a187-d6de7923ea8f service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state stopped and task_state powering-on.
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Successfully updated port: 65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:55 user nova-compute[71323]: INFO nova.compute.manager [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Rebuilding instance
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'trusted_certs' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_requests' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-e99be0b2-4a16-49d7-95ed-51790df3da10. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'migration_context' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Trying to apply a migration context that does not seem to be set for this instance {{(pid=71323) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.164s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:55 user nova-compute[71323]: WARNING nova.compute.manager [req-a9841919-d8ec-4ae0-ac99-744f90e94878 req-b785a460-756c-4dae-a84e-f476be2fc13d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.187s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance destroyed successfully.
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.205s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.compute.manager [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Detaching volume f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:56 user nova-compute[71323]: WARNING nova.compute.manager [req-6ff613b0-6371-4664-916f-97059b55316a req-ae27ec28-00df-43f6-9b34-6e0f192cb266 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state stopped and task_state powering-on.
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.virt.block_device [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attempting to driver detach volume f0c626e1-cd60-46d3-b894-a000a5cb3811 from mountpoint /dev/vdb
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Attempting to detach device vdb from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:01:56 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:01:56 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 03:01:56 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:01:56 user nova-compute[71323]:   <serial>f0c626e1-cd60-46d3-b894-a000a5cb3811</serial>
Apr 14 03:01:56 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:01:56 user nova-compute[71323]: </disk>
Apr 14 03:01:56 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441316.8769288, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Resumed (Lifecycle Event)
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance rebooted successfully.
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcd0766c-29b0-4f2c-aad8-e43165f8fdfb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully detached device vdb from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the persistent domain config.
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:56 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.010s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6711503e-08a4-4d1f-981e-131ac8088b0c]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:01:56 user nova-compute[71323]: No targets found.
Apr 14 03:01:56 user nova-compute[71323]: iSNS:
Apr 14 03:01:56 user nova-compute[71323]: No targets found.
Apr 14 03:01:56 user nova-compute[71323]: STATIC:
Apr 14 03:01:56 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:01:56 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:56 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:56 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:01:56 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:56 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:56 user nova-compute[71323]: FIRMWARE:
Apr 14 03:01:56 user nova-compute[71323]: No targets found.
Apr 14 03:01:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:01:56 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:01:56 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd5f2f0e-2562-4715-8594-a6617bf383cc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] During sync_power_state the instance has a pending task (powering-on). Skip.
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441316.877882, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:01:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Started (Lifecycle Event)
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[78cd4df8-de52-4175-a69f-9a277f1af073]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:57 user nova-compute[71323]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:01:57 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:57 user nova-compute[71323]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:01:57 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cda29e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b623d5c1-f020-40b6-a004-8ee605922a1b]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcf95549-533d-4593-a7b9-54eb15b971d2]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[69192d20-3c5a-4e55-9b32-78f570ed11db]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:01:57 user nova-compute[71323]: INFO nova.compute.claims [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Claim successful on node user
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.041s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[42aa481f-ae3c-48f8-ba7c-5ca16c86e81e]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f355c875-ca79-4695-ade1-f3119f1befe6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3e87374-44f4-4978-8acc-e338c086ebc9]: (4, ('Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]\nLogout of [sid: 9, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]
Apr 14 03:01:57 user nova-compute[71323]: Logout of [sid: 9, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.
Apr 14 03:01:57 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b17649d5-9ca4-4845-9286-8c4696d5add6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.494s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (504ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.718s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.compute.manager [req-5a4166be-c11d-4044-9bc4-190045c42af1 req-fa616c13-eaf5-4c19-bacc-a22f616ebd6f service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuilding.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:01:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.policy [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f089c5c093754f8082181a8fc1c002f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f987064bf0ed4a2fa8714064fe8b43c8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:01:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Creating image(s)
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "/opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "/opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "/opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8002MB free_disk=26.680397033691406GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance destroyed successfully.
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap04b5c1f1-89, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:01:58 user nova-compute[71323]: INFO os_vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:01:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:58 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[995ae9ff-91da-4e04-820c-cab48f52df92]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:01:58 user nova-compute[71323]: No targets found.
Apr 14 03:01:58 user nova-compute[71323]: iSNS:
Apr 14 03:01:58 user nova-compute[71323]: No targets found.
Apr 14 03:01:58 user nova-compute[71323]: STATIC:
Apr 14 03:01:58 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:01:58 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:01:58 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:01:58 user nova-compute[71323]: FIRMWARE:
Apr 14 03:01:58 user nova-compute[71323]: No targets found.
Apr 14 03:01:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:01:58 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:01:58 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a37c47d5-62b4-44a8-ba71-5b41d8414f41]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae431768-b0c9-4a1b-b8e4-8c872432b2cb]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:01:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd6950>, {}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.158s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (160ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:01:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Deleting instance files /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_del
Apr 14 03:01:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Deletion of /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_del complete
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 287a236d-d191-465f-bb65-636732ce5689 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance a78d15ea-7cb2-46e9-af86-8982e053d55a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.193s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk 1073741824" returned: 0 in 0.064s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.262s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: INFO nova.virt.block_device [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Booting with volume f0c626e1-cd60-46d3-b894-a000a5cb3811 at /dev/vdb
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.155s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Checking if we can resize image /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.utils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[664320c8-9155-45f6-9482-5befb9af04c5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e54da391-847c-4a79-8e83-89f89c3d82e3]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a3ba25d-7b31-4076-8530-b073f169cc52]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fbe88dd-1963-4451-a2c6-9ff9cd566726]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.238s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Cannot resize image /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'migration_context' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:01:59 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd210a4e-c1a3-4763-8636-b0a898cd0ed0]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG os_brick.utils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (182ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating existing volume attachment record: b82136c5-0a81-44df-924e-153ad5a7bedd {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Ensure instance console log exists: /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.834s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance network_info: |[{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:01:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Successfully created port: b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updated VIF entry in instance network info cache for port e99be0b2-4a16-49d7-95ed-51790df3da10. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-65202a6a-2d5f-420b-a2bb-02ad01831633. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:02:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Creating image(s)
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Ensure instance console log exists: /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Start _get_guest_xml network_info=[{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '7bc346aa-0798-431e-9e3c-58701616304e', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VlDJEXwv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'a78d15ea-7cb2-46e9-af86-8982e053d55a', 'attached_at': '', 'detached_at': '', 'volume_id': 'ba0a016e-f7ab-4aac-8fb3-ecf199678570', 'serial': 'ba0a016e-f7ab-4aac-8fb3-ecf199678570'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None}, {'device_type': 'disk', 'attachment_id': '897df238-c72c-46c7-80f3-6844d072066a', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KgfYmwnl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'a78d15ea-7cb2-46e9-af86-8982e053d55a', 'attached_at': '', 'detached_at': '', 'volume_id': '453b7855-8efd-41bd-806c-18ae2c03ac49', 'serial': '453b7855-8efd-41bd-806c-18ae2c03ac49'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': 1, 'volume_type': None}, {'device_type': 'disk', 'attachment_id': 'd1db064f-367e-4d97-a3dc-7007a652c17b', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pGxcVBXE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'a78d15ea-7cb2-46e9-af86-8982e053d55a', 'attached_at': '', 'detached_at': '', 'volume_id': 'c11ac81e-755b-4855-85b5-259c815fef2c', 'serial': 'c11ac81e-755b-4855-85b5-259c815fef2c'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdc', 'delete_on_termination': False, 'boot_index': 2, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VlDJEXwv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:00 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3179a468-104a-4718-9c77-2a0d5f073303]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:02:00 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f95aa42f-c83a-4198-bdba-89ac567b9945]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570] added
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f59b345c-5617-4b33-9cf8-695c73f39ce9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0378fc80-8007-496b-af68-60396806b5d6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VlDJEXwv {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VlDJEXwv" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[68af9f69-8ebf-4d16-8ea7-6dfeccaf1776]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VlDJEXwv'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4000309-1cbd-44e9-8252-4d6c9662b476]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updated VIF entry in instance network info cache for port 65202a6a-2d5f-420b-a2bb-02ad01831633. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[548dc5d2-fc01-4cfc-b4e7-742874566cf8]: (4, ('tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6b3e0b68-d3a4-48d1-b8be-ac57c39642f5 req-dd40d4b0-35a8-4836-a5b7-01661530e931 service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --login" returned: 0 in 0.069s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b385a4ed-8585-4eee-8e62-8b981ee08413]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260]
Apr 14 03:02:00 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260] successful.
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[02ca5cb0-18ce-4a7d-a842-2092b8f5ce50]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8bf5f90-b580-4c3f-b19f-95995e24220f]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.363s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] HCTL ('34', '-', '-', 0) found on session 10 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:02:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[238d78f7-2357-4d2f-a0fa-9d4ea41c14e1]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Searching for a device in session 10 and hctl ['34', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Successfully updated port: b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquired lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-changed-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Refreshing instance network info cache due to event network-changed-b975970e-4ce0-4b1c-b1b3-956befa892e0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] Acquiring lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updating instance_info_cache with network_info: [{"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Releasing lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Instance network_info: |[{"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] Acquired lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.neutron [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Refreshing network info cache for port b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Start _get_guest_xml network_info=[{"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:01 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:01 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1072264904',display_name='tempest-TestVolumeSwap-server-1072264904',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1072264904',id=14,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBnYqF9Yc0//Xc2kqydIgMZFcAkhLZn1nUjuDCDft78aomgzO7Y5n1Hl9qALLn2itHKhHsgBqDZqHr/4EY/d5MXpZoR4+r1CWeR2M7mZVFWbBOGHV8B0e9m5AsRBcS+9/g==',key_name='tempest-keypair-603146841',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f987064bf0ed4a2fa8714064fe8b43c8',ramdisk_id='',reservation_id='r-qoounn30',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-1328008515',owner_user_name='tempest-TestVolumeSwap-1328008515-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f089c5c093754f8082181a8fc1c002f4',uuid=b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converting VIF {"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'pci_devices' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:01 user nova-compute[71323]:   <uuid>b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49</uuid>
Apr 14 03:02:01 user nova-compute[71323]:   <name>instance-0000000e</name>
Apr 14 03:02:01 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:01 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:01 user nova-compute[71323]:   <metadata>
Apr 14 03:02:01 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:01 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:name>tempest-TestVolumeSwap-server-1072264904</nova:name>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:01</nova:creationTime>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:01 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:01 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:user uuid="f089c5c093754f8082181a8fc1c002f4">tempest-TestVolumeSwap-1328008515-project-member</nova:user>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:project uuid="f987064bf0ed4a2fa8714064fe8b43c8">tempest-TestVolumeSwap-1328008515</nova:project>
Apr 14 03:02:01 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:02:01 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:01 user nova-compute[71323]:         <nova:port uuid="b975970e-4ce0-4b1c-b1b3-956befa892e0">
Apr 14 03:02:01 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 03:02:01 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:01 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:01 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:01 user nova-compute[71323]:   </metadata>
Apr 14 03:02:01 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:01 user nova-compute[71323]:     <system>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="serial">b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49</entry>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="uuid">b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49</entry>
Apr 14 03:02:01 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:01 user nova-compute[71323]:     </system>
Apr 14 03:02:01 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:01 user nova-compute[71323]:   <os>
Apr 14 03:02:01 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:01 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:02:01 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:01 user nova-compute[71323]:   </os>
Apr 14 03:02:01 user nova-compute[71323]:   <features>
Apr 14 03:02:01 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:01 user nova-compute[71323]:     <apic/>
Apr 14 03:02:01 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:01 user nova-compute[71323]:   </features>
Apr 14 03:02:01 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:01 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:01 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:01 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:01 user nova-compute[71323]:   </clock>
Apr 14 03:02:01 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:01 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:01 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:01 user nova-compute[71323]:   </cpu>
Apr 14 03:02:01 user nova-compute[71323]:   <devices>
Apr 14 03:02:01 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:02:01 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:02:01 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk"/>
Apr 14 03:02:01 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:01 user nova-compute[71323]:     </disk>
Apr 14 03:02:01 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:01 user nova-compute[71323]:       <mac address="fa:16:3e:2b:e1:ca"/>
Apr 14 03:02:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:01 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:01 user nova-compute[71323]:       <target dev="tapb975970e-4c"/>
Apr 14 03:02:01 user nova-compute[71323]:     </interface>
Apr 14 03:02:01 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:01 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/console.log" append="off"/>
Apr 14 03:02:01 user nova-compute[71323]:     </serial>
Apr 14 03:02:01 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:01 user nova-compute[71323]:     <video>
Apr 14 03:02:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:01 user nova-compute[71323]:     </video>
Apr 14 03:02:01 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:01 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:01 user nova-compute[71323]:     </rng>
Apr 14 03:02:01 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:01 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:01 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:01 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:01 user nova-compute[71323]:   </devices>
Apr 14 03:02:01 user nova-compute[71323]: </domain>
Apr 14 03:02:01 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1072264904',display_name='tempest-TestVolumeSwap-server-1072264904',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1072264904',id=14,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBnYqF9Yc0//Xc2kqydIgMZFcAkhLZn1nUjuDCDft78aomgzO7Y5n1Hl9qALLn2itHKhHsgBqDZqHr/4EY/d5MXpZoR4+r1CWeR2M7mZVFWbBOGHV8B0e9m5AsRBcS+9/g==',key_name='tempest-keypair-603146841',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f987064bf0ed4a2fa8714064fe8b43c8',ramdisk_id='',reservation_id='r-qoounn30',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-1328008515',owner_user_name='tempest-TestVolumeSwap-1328008515-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f089c5c093754f8082181a8fc1c002f4',uuid=b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converting VIF {"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG os_vif [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb975970e-4c, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb975970e-4c, col_values=(('external_ids', {'iface-id': 'b975970e-4ce0-4b1c-b1b3-956befa892e0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2b:e1:ca', 'vm-uuid': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:01 user nova-compute[71323]: INFO os_vif [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c')
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] No VIF found with MAC fa:16:3e:2b:e1:ca, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Searching for a device in session 10 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VlDJEXwv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.393s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== connect_volume: return (1394ms) {'type': 'block', 'scsi_wwn': '26136363564353162', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26136363564353162', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KgfYmwnl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:02 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:02 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:02 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e06294e3-4524-4dde-8fdf-d74793ec0727]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:02:02 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[af383612-f37a-40c9-8945-1b296e255490]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49] added
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbbc9445-e7c9-4b18-aa36-2172514d88eb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f9aeca1-0d5b-406e-a0c3-77ad2caf2f2d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KgfYmwnl {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KgfYmwnl" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff21d5be-0d5b-48fa-8e21-dea7df8c3863]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KgfYmwnl'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updated VIF entry in instance network info cache for port b975970e-4ce0-4b1c-b1b3-956befa892e0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updating instance_info_cache with network_info: [{"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4648f02c-a3fa-4e08-90c8-9a55925fc6b9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-11adf592-3253-48ac-8c64-5857895578c0 req-deea42d0-3fde-4e9f-b67e-af605bf4345f service nova] Releasing lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c85813dd-a749-4ba9-bdde-ff3fcc4a8ad4]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[793015f2-cfb4-4b6c-8a28-ccf326bb0cd1]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260]
Apr 14 03:02:02 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260] successful.
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[02ec7919-8e72-4b74-bd34-389a6dce05a3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c83f67e2-fe35-4dbf-9f1e-b2b032aa28cb]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:02 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.295s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] HCTL ('35', '-', '-', 0) found on session 11 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de4e8d0f-0a84-4ad9-971c-8b30feb4a78e]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Searching for a device in session 11 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KgfYmwnl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] No waiting events found dispatching network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:03 user nova-compute[71323]: WARNING nova.compute.manager [req-265eb707-f5db-4032-9b49-38938960ff16 req-f1a3274f-6653-4f01-8dbd-3020dd5cbadf service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received unexpected event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 for instance with vm_state building and task_state spawning.
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.333s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== connect_volume: return (1335ms) {'type': 'block', 'scsi_wwn': '26135343664376261', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26135343664376261', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pGxcVBXE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:03 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:03 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:03 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[05c00e9f-34aa-4591-ae1c-c07af0de6e35]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:02:03 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[648cdf70-c4a4-432b-816f-a598cf72afa9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c] added
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9833349e-cd4c-495c-b09e-61a335552563]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[92b25f00-a7b0-45b2-a414-ccf9356bfeff]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pGxcVBXE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pGxcVBXE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[618774d4-c6f5-41e8-b41a-64fe5dbf9da9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pGxcVBXE'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5c3ee27-ae01-4582-b151-058d5e663947]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eefbc9c5-9d83-4c96-922c-5334578ed446]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --login" returned: 0 in 0.102s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a5c6006-0794-46c6-a62f-e39c91697a59]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260]
Apr 14 03:02:03 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260] successful.
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[95d71a1d-d557-4a06-850b-0fd30574fce7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[818943d0-7521-4d42-958c-c5f4f95cd5f8]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:03 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.392s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] HCTL ('36', '-', '-', 0) found on session 12 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0b9fb29-5715-4602-823e-6bcd753ff733]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Searching for a device in session 12 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pGxcVBXE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:02:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating image(s)
Apr 14 03:02:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.475s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== connect_volume: return (1478ms) {'type': 'block', 'scsi_wwn': '23036356436323332', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23036356436323332', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lazy-loading 'pci_devices' on Instance uuid a78d15ea-7cb2-46e9-af86-8982e053d55a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:04 user nova-compute[71323]:   <uuid>a78d15ea-7cb2-46e9-af86-8982e053d55a</uuid>
Apr 14 03:02:04 user nova-compute[71323]:   <name>instance-0000000d</name>
Apr 14 03:02:04 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:04 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:04 user nova-compute[71323]:   <metadata>
Apr 14 03:02:04 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:04 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:name>tempest-device-tagging-server-555989941</nova:name>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:00</nova:creationTime>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:04 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:04 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:user uuid="c3c342144e51496190e32329225ddd2b">tempest-TaggedBootDevicesTest-226254012-project-member</nova:user>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:project uuid="de2d10e0d7e7477484e4204841a3980c">tempest-TaggedBootDevicesTest-226254012</nova:project>
Apr 14 03:02:04 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:02:04 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="7e8d8948-ff3b-461a-921a-d90f46d9f84c">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="5b8574e2-f175-4432-a1cb-f17916ddc806">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.150" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="3e1675a7-2e92-4b45-a657-f7b811975097">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.34" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="04ec9de5-1982-4641-8e4f-6eed7f00cf51">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.132" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="fa3f5b14-6337-4e66-b990-1ba1cb8ab667">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.78" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="e99be0b2-4a16-49d7-95ed-51790df3da10">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:         <nova:port uuid="65202a6a-2d5f-420b-a2bb-02ad01831633">
Apr 14 03:02:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 14 03:02:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:04 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:04 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:04 user nova-compute[71323]:   </metadata>
Apr 14 03:02:04 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:04 user nova-compute[71323]:     <system>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="serial">a78d15ea-7cb2-46e9-af86-8982e053d55a</entry>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="uuid">a78d15ea-7cb2-46e9-af86-8982e053d55a</entry>
Apr 14 03:02:04 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:04 user nova-compute[71323]:     </system>
Apr 14 03:02:04 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:04 user nova-compute[71323]:   <os>
Apr 14 03:02:04 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:04 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:02:04 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:04 user nova-compute[71323]:   </os>
Apr 14 03:02:04 user nova-compute[71323]:   <features>
Apr 14 03:02:04 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:04 user nova-compute[71323]:     <apic/>
Apr 14 03:02:04 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:04 user nova-compute[71323]:   </features>
Apr 14 03:02:04 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:04 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:04 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:04 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:04 user nova-compute[71323]:   </clock>
Apr 14 03:02:04 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:04 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:04 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:04 user nova-compute[71323]:   </cpu>
Apr 14 03:02:04 user nova-compute[71323]:   <devices>
Apr 14 03:02:04 user nova-compute[71323]:     <disk type="file" device="cdrom">
Apr 14 03:02:04 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 03:02:04 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/disk.config"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="hda" bus="ide"/>
Apr 14 03:02:04 user nova-compute[71323]:     </disk>
Apr 14 03:02:04 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:02:04 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:02:04 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <serial>ba0a016e-f7ab-4aac-8fb3-ecf199678570</serial>
Apr 14 03:02:04 user nova-compute[71323]:     </disk>
Apr 14 03:02:04 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:02:04 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:02:04 user nova-compute[71323]:       <source dev="/dev/sdd"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <serial>453b7855-8efd-41bd-806c-18ae2c03ac49</serial>
Apr 14 03:02:04 user nova-compute[71323]:     </disk>
Apr 14 03:02:04 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:02:04 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:02:04 user nova-compute[71323]:       <source dev="/dev/sde"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="vdc" bus="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <serial>c11ac81e-755b-4855-85b5-259c815fef2c</serial>
Apr 14 03:02:04 user nova-compute[71323]:     </disk>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:e0:84:d8"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tap7e8d8948-ff"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:b1:4c:7d"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tap5b8574e2-f1"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:70:4f:8a"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tap3e1675a7-2e"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:a6:dd:c9"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tap04ec9de5-19"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:bd:ea:26"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tapfa3f5b14-63"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:08:0b:91"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tape99be0b2-4a"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:04 user nova-compute[71323]:       <mac address="fa:16:3e:c0:08:f4"/>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:04 user nova-compute[71323]:       <target dev="tap65202a6a-2d"/>
Apr 14 03:02:04 user nova-compute[71323]:     </interface>
Apr 14 03:02:04 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:04 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/console.log" append="off"/>
Apr 14 03:02:04 user nova-compute[71323]:     </serial>
Apr 14 03:02:04 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:04 user nova-compute[71323]:     <video>
Apr 14 03:02:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:04 user nova-compute[71323]:     </video>
Apr 14 03:02:04 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:04 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:04 user nova-compute[71323]:     </rng>
Apr 14 03:02:04 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:04 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:04 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:04 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:04 user nova-compute[71323]:   </devices>
Apr 14 03:02:04 user nova-compute[71323]: </domain>
Apr 14 03:02:04 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7e8d8948-ff, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap7e8d8948-ff, col_values=(('external_ids', {'iface-id': '7e8d8948-ff3b-461a-921a-d90f46d9f84c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e0:84:d8', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5b8574e2-f1, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap5b8574e2-f1, col_values=(('external_ids', {'iface-id': '5b8574e2-f175-4432-a1cb-f17916ddc806', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b1:4c:7d', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3e1675a7-2e, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3e1675a7-2e, col_values=(('external_ids', {'iface-id': '3e1675a7-2e92-4b45-a657-f7b811975097', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:70:4f:8a', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap04ec9de5-19, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap04ec9de5-19, col_values=(('external_ids', {'iface-id': '04ec9de5-1982-4641-8e4f-6eed7f00cf51', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:a6:dd:c9', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfa3f5b14-63, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfa3f5b14-63, col_values=(('external_ids', {'iface-id': 'fa3f5b14-6337-4e66-b990-1ba1cb8ab667', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:bd:ea:26', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk 1073741824" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.205s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape99be0b2-4a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape99be0b2-4a, col_values=(('external_ids', {'iface-id': 'e99be0b2-4a16-49d7-95ed-51790df3da10', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:08:0b:91', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap65202a6a-2d, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap65202a6a-2d, col_values=(('external_ids', {'iface-id': '65202a6a-2d5f-420b-a2bb-02ad01831633', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c0:08:f4', 'vm-uuid': 'a78d15ea-7cb2-46e9-af86-8982e053d55a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] No waiting events found dispatching network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:05 user nova-compute[71323]: WARNING nova.compute.manager [req-d2ffc9b5-3a5a-4afa-a0ee-583400f8fb64 req-46db8be6-4e33-40a4-b2a0-7f8c0661b230 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received unexpected event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 for instance with vm_state building and task_state spawning.
Apr 14 03:02:05 user nova-compute[71323]: INFO os_vif [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d')
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441325.4460099, b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] VM Resumed (Lifecycle Event)
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Instance spawned successfully.
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441325.4464097, b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] VM Started (Lifecycle Event)
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Cannot resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Ensure instance console log exists: /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start _get_guest_xml network_info=[{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'b82136c5-0a81-44df-924e-153ad5a7bedd', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4', 'attached_at': '', 'detached_at': '', 'volume_id': 'f0c626e1-cd60-46d3-b894-a000a5cb3811', 'serial': 'f0c626e1-cd60-46d3-b894-a000a5cb3811'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] No BDM found with device name hda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] No VIF found with MAC fa:16:3e:e0:84:d8, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] No VIF found with MAC fa:16:3e:bd:ea:26, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Using config drive
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 14 03:02:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Took 6.91 seconds to spawn the instance on the hypervisor.
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:05 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[43c25b3d-9edb-458e-a60a-3348f01f8616]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:02:05 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[267cce59-ae60-43c6-90e0-fd7328a6353f]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811] added
Apr 14 03:02:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[280860df-539f-481c-9c32-c79e1306a808]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.compute.manager [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Took 8.55 seconds to build instance.
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efb1a9a0-e6fb-4a1d-808a-f9d3ee4053ab]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4faee412-c7f5-4724-a362-293239d21f4c tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.703s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cpDvVdUM {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Creating config drive at /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/disk.config
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp5n0yvvvu {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cpDvVdUM" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccbc88db-3768-4e69-8fb9-2957be1c4a66]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cpDvVdUM'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp5n0yvvvu" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[178c6ffe-2fe3-42d3-849c-51f94db07f72]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b69c267-e78c-4c4c-a414-8d11906e0b07]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:05 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --login" returned: 0 in 0.084s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[72cb7f27-af36-4769-8045-415a11d6edcf]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]
Apr 14 03:02:05 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.
Apr 14 03:02:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0255ca9-a6be-45c1-b280-bea1047ceac4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6dfbb302-d19d-42ef-8c85-5da087c59a4e]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.390s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('37', '-', '-', 0) found on session 13 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2a75b14-c123-40be-a44f-60c6ea4fb06f]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 13 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.428s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1430ms) {'type': 'block', 'scsi_wwn': '26336373965373062', 'path': '/dev/sdf'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26336373965373062', 'path': '/dev/sdf'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:07 user nova-compute[71323]:   <uuid>6bb082bc-0c74-4149-9bd0-ceb09d7764d4</uuid>
Apr 14 03:02:07 user nova-compute[71323]:   <name>instance-00000005</name>
Apr 14 03:02:07 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:07 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:07 user nova-compute[71323]:   <metadata>
Apr 14 03:02:07 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:07 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-1052167823</nova:name>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:05</nova:creationTime>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:07 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:07 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:02:07 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:02:07 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:07 user nova-compute[71323]:         <nova:port uuid="04b5c1f1-8988-4a34-b97a-ad0cecd12b71">
Apr 14 03:02:07 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:02:07 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:07 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:07 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:07 user nova-compute[71323]:   </metadata>
Apr 14 03:02:07 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:07 user nova-compute[71323]:     <system>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="serial">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="uuid">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:02:07 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:07 user nova-compute[71323]:     </system>
Apr 14 03:02:07 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:07 user nova-compute[71323]:   <os>
Apr 14 03:02:07 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:07 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:02:07 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:07 user nova-compute[71323]:   </os>
Apr 14 03:02:07 user nova-compute[71323]:   <features>
Apr 14 03:02:07 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:07 user nova-compute[71323]:     <apic/>
Apr 14 03:02:07 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:07 user nova-compute[71323]:   </features>
Apr 14 03:02:07 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:07 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:07 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:07 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:07 user nova-compute[71323]:   </clock>
Apr 14 03:02:07 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:07 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:07 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:07 user nova-compute[71323]:   </cpu>
Apr 14 03:02:07 user nova-compute[71323]:   <devices>
Apr 14 03:02:07 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:02:07 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:02:07 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk"/>
Apr 14 03:02:07 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:07 user nova-compute[71323]:     </disk>
Apr 14 03:02:07 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:02:07 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:02:07 user nova-compute[71323]:       <source dev="/dev/sdf"/>
Apr 14 03:02:07 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:02:07 user nova-compute[71323]:       <serial>f0c626e1-cd60-46d3-b894-a000a5cb3811</serial>
Apr 14 03:02:07 user nova-compute[71323]:     </disk>
Apr 14 03:02:07 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:07 user nova-compute[71323]:       <mac address="fa:16:3e:af:bb:dd"/>
Apr 14 03:02:07 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:07 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:07 user nova-compute[71323]:       <target dev="tap04b5c1f1-89"/>
Apr 14 03:02:07 user nova-compute[71323]:     </interface>
Apr 14 03:02:07 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:07 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log" append="off"/>
Apr 14 03:02:07 user nova-compute[71323]:     </serial>
Apr 14 03:02:07 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:07 user nova-compute[71323]:     <video>
Apr 14 03:02:07 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:07 user nova-compute[71323]:     </video>
Apr 14 03:02:07 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:07 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:07 user nova-compute[71323]:     </rng>
Apr 14 03:02:07 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:07 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:07 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:07 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:07 user nova-compute[71323]:   </devices>
Apr 14 03:02:07 user nova-compute[71323]: </domain>
Apr 14 03:02:07 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:01:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG os_vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap04b5c1f1-89, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap04b5c1f1-89, col_values=(('external_ids', {'iface-id': '04b5c1f1-8988-4a34-b97a-ad0cecd12b71', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:af:bb:dd', 'vm-uuid': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: INFO os_vif [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:af:bb:dd, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:07 user nova-compute[71323]: WARNING nova.compute.manager [req-e9c5d5b4-1442-4bac-b6ef-446e3127a2fe req-9d95d27b-0aa4-4383-8067-07e78970a7af service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c for instance with vm_state building and task_state spawning.
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:08 user nova-compute[71323]: WARNING nova.compute.manager [req-4d825659-e57e-4639-80e9-7ea06490e518 req-e06dba0f-ee8c-4791-8491-f765871ae03c service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 for instance with vm_state building and task_state spawning.
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c for instance with vm_state building and task_state spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-871870e8-c595-45a3-834e-acd124efbcf7 req-95c18028-cb0f-446f-add1-67e5f076dc64 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 for instance with vm_state building and task_state spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 for instance with vm_state building and task_state spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 for instance with vm_state building and task_state spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-af3eb04d-d40c-4f18-aa2a-efc82b73f9a5 req-72313aa8-f15b-4ca2-a529-8e18512b501d service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 for instance with vm_state building and task_state spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:10 user nova-compute[71323]: WARNING nova.compute.manager [req-5ea19de2-c281-437b-b17f-3309b3f8eadb req-83858a0c-3c58-42d5-b7e7-566bb573a55d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuild_spawning.
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:11 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441316.5181298, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:11 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Stopped (Lifecycle Event)
Apr 14 03:02:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-748781ba-3897-4acc-942d-517773278b1f None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-a389d6b8-0a6a-457d-89ef-2fece4293277 req-6ebbfaf5-9153-44e8-bff9-a5aaa99f7d14 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-272c7135-df7b-458b-a6e8-f1d956fcee4e req-ad8b3632-3e86-472d-b407-b6fedc23686f service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 for instance with vm_state building and task_state spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:12 user nova-compute[71323]: WARNING nova.compute.manager [req-1fed7410-04b1-416e-b416-e4daa8825cb4 req-55213fb0-bd89-45d2-8f87-0fb94ca3f3dc service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state rebuild_spawning.
Apr 14 03:02:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441333.4298859, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Resumed (Lifecycle Event)
Apr 14 03:02:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance spawned successfully.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441333.4400277, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Started (Lifecycle Event)
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (rebuild_spawning). Skip.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441333.5974302, a78d15ea-7cb2-46e9-af86-8982e053d55a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] VM Resumed (Lifecycle Event)
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Trying to apply a migration context that does not seem to be set for this instance {{(pid=71323) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1032}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance spawned successfully.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441333.6022286, a78d15ea-7cb2-46e9-af86-8982e053d55a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] VM Started (Lifecycle Event)
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b8231917-215f-42f7-b3f3-2057fb1c602b tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.099s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Took 13.24 seconds to spawn the instance on the hypervisor.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:13 user nova-compute[71323]: INFO nova.compute.manager [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Took 30.31 seconds to build instance.
Apr 14 03:02:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-33369a24-11d9-40ea-b360-1ce5344f0d6b tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 30.400s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:14 user nova-compute[71323]: WARNING nova.compute.manager [req-b323598c-4551-4f24-be2c-0588eaaad5ef req-c9145dfa-97c8-40cd-8d77-6c3aba0a32c1 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 for instance with vm_state active and task_state None.
Apr 14 03:02:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:21 user nova-compute[71323]: INFO nova.compute.manager [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] instance snapshotting
Apr 14 03:02:21 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Beginning live snapshot process
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json -f qcow2" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json -f qcow2" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995.delta 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995.delta 1073741824" returned: 0 in 0.062s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 03:02:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] COPY block job progress, current cursor: 0 final cursor: 43843584 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:02:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] COPY block job progress, current cursor: 43843584 final cursor: 43843584 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:02:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 03:02:23 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:02:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995.delta /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995.delta /opt/stack/data/nova/instances/snapshots/tmp68u23ytw/f01bb6952d0b4a1196e8632bb7803995" returned: 0 in 0.355s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Snapshot extracted, beginning image upload
Apr 14 03:02:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Snapshot image upload complete
Apr 14 03:02:27 user nova-compute[71323]: INFO nova.compute.manager [None req-51f4c31b-71b8-4557-859f-01ce25319082 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Took 5.56 seconds to snapshot the instance on the hypervisor.
Apr 14 03:02:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:02:30 user nova-compute[71323]: INFO nova.compute.claims [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Claim successful on node user
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.463s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:02:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:02:30 user nova-compute[71323]: INFO nova.virt.block_device [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Booting with volume 37bd0d93-b922-426c-8986-d86e886da69e at /dev/vda
Apr 14 03:02:30 user nova-compute[71323]: DEBUG nova.policy [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG os_brick.utils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee161993-640a-4dcd-bfdb-6b7290175982]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[46d9b9f7-65fd-40d7-86ec-c3ae7dad5922]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[99d35902-9844-4459-b293-293f18e56a51]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0b4d5d4-e003-4360-b3fc-81fd68a11ac5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:02:31 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:02:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cecccfb0-d297-47df-a90b-1ff79ef222d2]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG os_brick.utils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updating existing volume attachment record: 1b28a89d-4544-49fc-afc0-487d95abdf46 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:02:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Successfully created port: e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Successfully updated port: e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-changed-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Refreshing instance network info cache due to event network-changed-e51353cf-a5e9-41e4-9b6e-4a022984af76. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] Acquiring lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:02:33 user nova-compute[71323]: INFO nova.compute.claims [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Claim successful on node user
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updating instance_info_cache with network_info: [{"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Instance network_info: |[{"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] Acquired lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Refreshing network info cache for port e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.648s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updated VIF entry in instance network info cache for port e51353cf-a5e9-41e4-9b6e-4a022984af76. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updating instance_info_cache with network_info: [{"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-540f95db-0c16-46bf-98a5-7223ec5ad24d req-c376ae15-4d46-4bca-ba61-70632c44e62c service nova] Releasing lock "refresh_cache-f970adb1-fe1b-4b58-83aa-45207a6211ef" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:02:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:02:33 user nova-compute[71323]: DEBUG nova.policy [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '337e2d8118324215b5a0baa36a5895b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6371b4c05df843da9b134e12994c1d7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:02:34 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Creating image(s)
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "/opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "/opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "/opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "f6ce21d329050bd2327ddb777b2496513fa23e76" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "f6ce21d329050bd2327ddb777b2496513fa23e76" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.part --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG nova.virt.images [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] a7190884-2b37-45b0-a8e1-4e55a042e5ea was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:02:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.part /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Successfully created port: a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.part /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.converted" returned: 0 in 0.232s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76.converted --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "f6ce21d329050bd2327ddb777b2496513fa23e76" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.312s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "f6ce21d329050bd2327ddb777b2496513fa23e76" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "f6ce21d329050bd2327ddb777b2496513fa23e76" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76,backing_fmt=raw /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76,backing_fmt=raw /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk 1073741824" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "f6ce21d329050bd2327ddb777b2496513fa23e76" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.198s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'migration_context' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Ensure instance console log exists: /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:02:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Creating image(s)
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Ensure instance console log exists: /opt/stack/data/nova/instances/f970adb1-fe1b-4b58-83aa-45207a6211ef/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Start _get_guest_xml network_info=[{"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '1b28a89d-4544-49fc-afc0-487d95abdf46', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YcTtkeKx', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'f970adb1-fe1b-4b58-83aa-45207a6211ef', 'attached_at': '', 'detached_at': '', 'volume_id': '37bd0d93-b922-426c-8986-d86e886da69e', 'serial': '37bd0d93-b922-426c-8986-d86e886da69e'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:36 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:36 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YcTtkeKx', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:02:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:02:36 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Successfully updated port: a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing instance network info cache due to event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] Acquiring lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] Acquired lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing network info cache for port a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[265cc2a6-3c16-464d-aa84-61682ca0ad37]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:02:36 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[70fa91a2-95c4-4652-9b11-61591a77e855]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e] added
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d98a3631-fc1e-41c0-9f33-13df8f28f481]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac95ee8b-b7a0-46b5-ab9a-2056f3092c85]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YcTtkeKx {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YcTtkeKx" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[abff8ce4-989a-487d-b943-6390a6a4fb5f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YcTtkeKx'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a15d7c46-86fc-4474-8f4d-2d768798ebd3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[268b5865-87aa-48cc-886e-d94e33813436]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e63d2353-0e6b-4f7e-8e46-8c82ea2f621e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260]
Apr 14 03:02:36 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260] successful.
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f4e4d079-10eb-40c6-859f-b9a100b4168b req-f08c18bc-ee39-442c-b335-d39ee2da2550 service nova] Releasing lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquired lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9f571be-c992-402b-b968-d38674938925]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[79539d55-1ab0-4fbd-81a2-6bee0860c0e0]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:02:36 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:02:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.349s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('38', '-', '-', 0) found on session 14 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d843cb36-6365-461e-b6f1-c89fa30ff3d6]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 14 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YcTtkeKx', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.network.neutron [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating instance_info_cache with network_info: [{"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Releasing lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance network_info: |[{"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Start _get_guest_xml network_info=[{"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='3b82065ba78a392ac2f9458a3b323247',container_format='bare',created_at=2023-04-14T03:02:21Z,direct_url=<?>,disk_format='qcow2',id=a7190884-2b37-45b0-a8e1-4e55a042e5ea,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-599123546',owner='6371b4c05df843da9b134e12994c1d7a',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-14T03:02:27Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': 'a7190884-2b37-45b0-a8e1-4e55a042e5ea'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:37 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:37 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='3b82065ba78a392ac2f9458a3b323247',container_format='bare',created_at=2023-04-14T03:02:21Z,direct_url=<?>,disk_format='qcow2',id=a7190884-2b37-45b0-a8e1-4e55a042e5ea,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-599123546',owner='6371b4c05df843da9b134e12994c1d7a',properties=ImageMetaProps,protected=<?>,size=38666240,status='active',tags=<?>,updated_at=2023-04-14T03:02:27Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-653907637',display_name='tempest-TestStampPattern-server-653907637',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-653907637',id=16,image_ref='a7190884-2b37-45b0-a8e1-4e55a042e5ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cc0ip46y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='2ca84990-b165-4d0b-a917-307cd8f96f28',image_min_disk='1',image_min_ram='0',image_owner_id='6371b4c05df843da9b134e12994c1d7a',image_owner_project_name='tempest-TestStampPattern-323884970',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-323884970-project-member',image_user_id='337e2d8118324215b5a0baa36a5895b0',network_allocated='True',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:02:34Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=dc8c2e40-f896-4d21-9927-67330485d77d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.objects.instance [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'pci_devices' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:37 user nova-compute[71323]:   <uuid>dc8c2e40-f896-4d21-9927-67330485d77d</uuid>
Apr 14 03:02:37 user nova-compute[71323]:   <name>instance-00000010</name>
Apr 14 03:02:37 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:37 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:37 user nova-compute[71323]:   <metadata>
Apr 14 03:02:37 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:37 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:name>tempest-TestStampPattern-server-653907637</nova:name>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:37</nova:creationTime>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:37 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:user uuid="337e2d8118324215b5a0baa36a5895b0">tempest-TestStampPattern-323884970-project-member</nova:user>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:project uuid="6371b4c05df843da9b134e12994c1d7a">tempest-TestStampPattern-323884970</nova:project>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:root type="image" uuid="a7190884-2b37-45b0-a8e1-4e55a042e5ea"/>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:port uuid="a36f717a-9a8b-43c9-ba01-0f03a522d166">
Apr 14 03:02:37 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:02:37 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:37 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:37 user nova-compute[71323]:   </metadata>
Apr 14 03:02:37 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:37 user nova-compute[71323]:     <system>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="serial">dc8c2e40-f896-4d21-9927-67330485d77d</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="uuid">dc8c2e40-f896-4d21-9927-67330485d77d</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:37 user nova-compute[71323]:     </system>
Apr 14 03:02:37 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:37 user nova-compute[71323]:   <os>
Apr 14 03:02:37 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:37 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:02:37 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:37 user nova-compute[71323]:   </os>
Apr 14 03:02:37 user nova-compute[71323]:   <features>
Apr 14 03:02:37 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:37 user nova-compute[71323]:     <apic/>
Apr 14 03:02:37 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:37 user nova-compute[71323]:   </features>
Apr 14 03:02:37 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:37 user nova-compute[71323]:   </clock>
Apr 14 03:02:37 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:37 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:37 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:37 user nova-compute[71323]:   </cpu>
Apr 14 03:02:37 user nova-compute[71323]:   <devices>
Apr 14 03:02:37 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:02:37 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:02:37 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk"/>
Apr 14 03:02:37 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:     </disk>
Apr 14 03:02:37 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:37 user nova-compute[71323]:       <mac address="fa:16:3e:ab:25:10"/>
Apr 14 03:02:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:37 user nova-compute[71323]:       <target dev="tapa36f717a-9a"/>
Apr 14 03:02:37 user nova-compute[71323]:     </interface>
Apr 14 03:02:37 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:37 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/console.log" append="off"/>
Apr 14 03:02:37 user nova-compute[71323]:     </serial>
Apr 14 03:02:37 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:37 user nova-compute[71323]:     <video>
Apr 14 03:02:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:     </video>
Apr 14 03:02:37 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:37 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:37 user nova-compute[71323]:     </rng>
Apr 14 03:02:37 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:37 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:37 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:37 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:37 user nova-compute[71323]:   </devices>
Apr 14 03:02:37 user nova-compute[71323]: </domain>
Apr 14 03:02:37 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-653907637',display_name='tempest-TestStampPattern-server-653907637',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-653907637',id=16,image_ref='a7190884-2b37-45b0-a8e1-4e55a042e5ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cc0ip46y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='2ca84990-b165-4d0b-a917-307cd8f96f28',image_min_disk='1',image_min_ram='0',image_owner_id='6371b4c05df843da9b134e12994c1d7a',image_owner_project_name='tempest-TestStampPattern-323884970',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-323884970-project-member',image_user_id='337e2d8118324215b5a0baa36a5895b0',network_allocated='True',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:02:34Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=dc8c2e40-f896-4d21-9927-67330485d77d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG os_vif [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa36f717a-9a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa36f717a-9a, col_values=(('external_ids', {'iface-id': 'a36f717a-9a8b-43c9-ba01-0f03a522d166', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ab:25:10', 'vm-uuid': 'dc8c2e40-f896-4d21-9927-67330485d77d'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: INFO os_vif [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a')
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No VIF found with MAC fa:16:3e:ab:25:10, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.394s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1397ms) {'type': 'block', 'scsi_wwn': '23736303162613862', 'path': '/dev/sdg'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23736303162613862', 'path': '/dev/sdg'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1182227426',id=15,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-vngr2krk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:02:31Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=f970adb1-fe1b-4b58-83aa-45207a6211ef,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.objects.instance [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid f970adb1-fe1b-4b58-83aa-45207a6211ef {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:37 user nova-compute[71323]:   <uuid>f970adb1-fe1b-4b58-83aa-45207a6211ef</uuid>
Apr 14 03:02:37 user nova-compute[71323]:   <name>instance-0000000f</name>
Apr 14 03:02:37 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:37 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:37 user nova-compute[71323]:   <metadata>
Apr 14 03:02:37 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:37 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-volume-backed-server-1182227426</nova:name>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:36</nova:creationTime>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:37 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:37 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:37 user nova-compute[71323]:         <nova:port uuid="e51353cf-a5e9-41e4-9b6e-4a022984af76">
Apr 14 03:02:37 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 03:02:37 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:37 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:37 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:37 user nova-compute[71323]:   </metadata>
Apr 14 03:02:37 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:37 user nova-compute[71323]:     <system>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="serial">f970adb1-fe1b-4b58-83aa-45207a6211ef</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="uuid">f970adb1-fe1b-4b58-83aa-45207a6211ef</entry>
Apr 14 03:02:37 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:37 user nova-compute[71323]:     </system>
Apr 14 03:02:37 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:37 user nova-compute[71323]:   <os>
Apr 14 03:02:37 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:37 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:02:37 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:37 user nova-compute[71323]:   </os>
Apr 14 03:02:37 user nova-compute[71323]:   <features>
Apr 14 03:02:37 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:37 user nova-compute[71323]:     <apic/>
Apr 14 03:02:37 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:37 user nova-compute[71323]:   </features>
Apr 14 03:02:37 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:37 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:37 user nova-compute[71323]:   </clock>
Apr 14 03:02:37 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:37 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:37 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:37 user nova-compute[71323]:   </cpu>
Apr 14 03:02:37 user nova-compute[71323]:   <devices>
Apr 14 03:02:37 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:02:37 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:02:37 user nova-compute[71323]:       <source dev="/dev/sdg"/>
Apr 14 03:02:37 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:       <serial>37bd0d93-b922-426c-8986-d86e886da69e</serial>
Apr 14 03:02:37 user nova-compute[71323]:     </disk>
Apr 14 03:02:37 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:37 user nova-compute[71323]:       <mac address="fa:16:3e:0c:70:d3"/>
Apr 14 03:02:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:37 user nova-compute[71323]:       <target dev="tape51353cf-a5"/>
Apr 14 03:02:37 user nova-compute[71323]:     </interface>
Apr 14 03:02:37 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:37 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/f970adb1-fe1b-4b58-83aa-45207a6211ef/console.log" append="off"/>
Apr 14 03:02:37 user nova-compute[71323]:     </serial>
Apr 14 03:02:37 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:37 user nova-compute[71323]:     <video>
Apr 14 03:02:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:37 user nova-compute[71323]:     </video>
Apr 14 03:02:37 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:37 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:37 user nova-compute[71323]:     </rng>
Apr 14 03:02:37 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:37 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:37 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:37 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:37 user nova-compute[71323]:   </devices>
Apr 14 03:02:37 user nova-compute[71323]: </domain>
Apr 14 03:02:37 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1182227426',id=15,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-vngr2krk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:02:31Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=f970adb1-fe1b-4b58-83aa-45207a6211ef,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG os_vif [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape51353cf-a5, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tape51353cf-a5, col_values=(('external_ids', {'iface-id': 'e51353cf-a5e9-41e4-9b6e-4a022984af76', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0c:70:d3', 'vm-uuid': 'f970adb1-fe1b-4b58-83aa-45207a6211ef'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:37 user nova-compute[71323]: INFO os_vif [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5')
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:0c:70:d3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] No waiting events found dispatching network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:38 user nova-compute[71323]: WARNING nova.compute.manager [req-47767dee-b000-44a3-a222-f1f227b0dd5d req-98eaafa2-8690-4d3f-92b1-eb4e1b07ec7d service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received unexpected event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 for instance with vm_state building and task_state spawning.
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] No waiting events found dispatching network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:40 user nova-compute[71323]: WARNING nova.compute.manager [req-3c7153f1-3f6c-4b31-a985-d441ca6ca470 req-bb3c4440-b4b4-4b9a-9eff-a2c1b88d65bc service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received unexpected event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 for instance with vm_state building and task_state spawning.
Apr 14 03:02:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] No waiting events found dispatching network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:40 user nova-compute[71323]: WARNING nova.compute.manager [req-6caa6191-7e3e-479b-922d-080bb6bb255b req-7017275a-71ec-4440-8969-00638d986479 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received unexpected event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 for instance with vm_state building and task_state spawning.
Apr 14 03:02:41 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441361.9787147, dc8c2e40-f896-4d21-9927-67330485d77d => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:41 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] VM Resumed (Lifecycle Event)
Apr 14 03:02:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:02:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:02:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance spawned successfully.
Apr 14 03:02:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:02:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441361.980464, dc8c2e40-f896-4d21-9927-67330485d77d => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] VM Started (Lifecycle Event)
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Took 8.03 seconds to spawn the instance on the hypervisor.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441362.1102598, f970adb1-fe1b-4b58-83aa-45207a6211ef => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] VM Resumed (Lifecycle Event)
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Instance spawned successfully.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Took 9.17 seconds to build instance.
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441362.112113, f970adb1-fe1b-4b58-83aa-45207a6211ef => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] VM Started (Lifecycle Event)
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-96eb799b-e92b-4fab-a5ac-cc2f24b73120 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.310s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] No waiting events found dispatching network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:42 user nova-compute[71323]: WARNING nova.compute.manager [req-77629780-4add-4bb8-9974-414a12607e89 req-e15c88c3-b79e-4c7d-9b93-120b55b7e56e service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received unexpected event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 for instance with vm_state building and task_state spawning.
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Took 5.97 seconds to spawn the instance on the hypervisor.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Took 12.12 seconds to build instance.
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-857e5d41-1823-4767-b53f-d73424560c86 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.242s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:42 user nova-compute[71323]: INFO nova.compute.manager [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Rescuing
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:43 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance destroyed successfully.
Apr 14 03:02:43 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Attempting rescue
Apr 14 03:02:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:02:43 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Creating image(s)
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'trusted_certs' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:43 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue" returned: 0 in 0.054s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.203s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start _get_guest_xml network_info=[{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:f3:f8:fa"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'numa_topology' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'vcpu_model' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:44 user nova-compute[71323]: WARNING nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state active and task_state rescuing.
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:44 user nova-compute[71323]: WARNING nova.compute.manager [req-575a231e-63bb-4417-840a-f582897f7659 req-1429653c-b10b-4bf9-ac13-2926d5d7a794 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state active and task_state rescuing.
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-846820438',display_name='tempest-ServerRescueNegativeTestJSON-server-846820438',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-846820438',id=3,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-5u5lcg43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T02:58:13Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=5e57cef1-b6cd-4291-a4c5-60e9dd22aed6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:f3:f8:fa"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:f3:f8:fa"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:02:44 user nova-compute[71323]:   <uuid>5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</uuid>
Apr 14 03:02:44 user nova-compute[71323]:   <name>instance-00000003</name>
Apr 14 03:02:44 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:02:44 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:02:44 user nova-compute[71323]:   <metadata>
Apr 14 03:02:44 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:02:44 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-846820438</nova:name>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:02:44</nova:creationTime>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:02:44 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:02:44 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:owner>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 03:02:44 user nova-compute[71323]:       </nova:owner>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:02:44 user nova-compute[71323]:       <nova:ports>
Apr 14 03:02:44 user nova-compute[71323]:         <nova:port uuid="1319e09d-3a9f-404b-862d-37b385badb8d">
Apr 14 03:02:44 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:02:44 user nova-compute[71323]:         </nova:port>
Apr 14 03:02:44 user nova-compute[71323]:       </nova:ports>
Apr 14 03:02:44 user nova-compute[71323]:     </nova:instance>
Apr 14 03:02:44 user nova-compute[71323]:   </metadata>
Apr 14 03:02:44 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:02:44 user nova-compute[71323]:     <system>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="serial">5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</entry>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="uuid">5e57cef1-b6cd-4291-a4c5-60e9dd22aed6</entry>
Apr 14 03:02:44 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:02:44 user nova-compute[71323]:     </system>
Apr 14 03:02:44 user nova-compute[71323]:   </sysinfo>
Apr 14 03:02:44 user nova-compute[71323]:   <os>
Apr 14 03:02:44 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:02:44 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:02:44 user nova-compute[71323]:   </os>
Apr 14 03:02:44 user nova-compute[71323]:   <features>
Apr 14 03:02:44 user nova-compute[71323]:     <acpi/>
Apr 14 03:02:44 user nova-compute[71323]:     <apic/>
Apr 14 03:02:44 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:02:44 user nova-compute[71323]:   </features>
Apr 14 03:02:44 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:02:44 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:02:44 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:02:44 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:02:44 user nova-compute[71323]:   </clock>
Apr 14 03:02:44 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:02:44 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:02:44 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:02:44 user nova-compute[71323]:   </cpu>
Apr 14 03:02:44 user nova-compute[71323]:   <devices>
Apr 14 03:02:44 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:02:44 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:02:44 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue"/>
Apr 14 03:02:44 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:02:44 user nova-compute[71323]:     </disk>
Apr 14 03:02:44 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:02:44 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:02:44 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk"/>
Apr 14 03:02:44 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:02:44 user nova-compute[71323]:     </disk>
Apr 14 03:02:44 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:02:44 user nova-compute[71323]:       <mac address="fa:16:3e:f3:f8:fa"/>
Apr 14 03:02:44 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:44 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:02:44 user nova-compute[71323]:       <target dev="tap1319e09d-3a"/>
Apr 14 03:02:44 user nova-compute[71323]:     </interface>
Apr 14 03:02:44 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:02:44 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/console.log" append="off"/>
Apr 14 03:02:44 user nova-compute[71323]:     </serial>
Apr 14 03:02:44 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:02:44 user nova-compute[71323]:     <video>
Apr 14 03:02:44 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:02:44 user nova-compute[71323]:     </video>
Apr 14 03:02:44 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:02:44 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:02:44 user nova-compute[71323]:     </rng>
Apr 14 03:02:44 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:02:44 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:02:44 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:02:44 user nova-compute[71323]:     </memballoon>
Apr 14 03:02:44 user nova-compute[71323]:   </devices>
Apr 14 03:02:44 user nova-compute[71323]: </domain>
Apr 14 03:02:44 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:02:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance destroyed successfully.
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:02:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:f3:f8:fa, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:02:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:46 user nova-compute[71323]: WARNING nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state active and task_state rescuing.
Apr 14 03:02:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:02:46 user nova-compute[71323]: WARNING nova.compute.manager [req-071d6d55-47b8-4555-9358-dc44123cdc93 req-3c76f636-a1d7-4558-9e38-8db5d4a08524 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state active and task_state rescuing.
Apr 14 03:02:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441368.0896194, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:48 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Resumed (Lifecycle Event)
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-37078be6-0085-4451-8582-9907b67567f1 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:48 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441368.0913992, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:02:48 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Started (Lifecycle Event)
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:02:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:02:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:02:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:02:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:02:53 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdc) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:02:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7237MB free_disk=26.537487030029297GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f970adb1-fe1b-4b58-83aa-45207a6211ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance dc8c2e40-f896-4d21-9927-67330485d77d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 287a236d-d191-465f-bb65-636732ce5689 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance a78d15ea-7cb2-46e9-af86-8982e053d55a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=2048MB phys_disk=40GB used_disk=10GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:02:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.565s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:06 user nova-compute[71323]: INFO nova.compute.manager [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Terminating instance
Apr 14 03:03:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-unplugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] No waiting events found dispatching network-vif-unplugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-ad8903af-c170-4439-9d98-b7a946963ac7 req-a0c8d9ec-6ac7-439a-af34-0dee074c1e5f service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-unplugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] Instance destroyed successfully.
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'resources' on Instance uuid 287a236d-d191-465f-bb65-636732ce5689 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-437824819',display_name='tempest-VolumesAdminNegativeTest-server-437824819',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-437824819',id=12,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:01:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-cwcb1gb7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:01:17Z,user_data=None,user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=287a236d-d191-465f-bb65-636732ce5689,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "address": "fa:16:3e:6b:95:3f", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7a4cd76a-76", "ovs_interfaceid": "7a4cd76a-7605-4209-9dd2-ec4bced26f20", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG os_vif [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7a4cd76a-76, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:07 user nova-compute[71323]: INFO os_vif [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6b:95:3f,bridge_name='br-int',has_traffic_filtering=True,id=7a4cd76a-7605-4209-9dd2-ec4bced26f20,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7a4cd76a-76')
Apr 14 03:03:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Deleting instance files /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689_del
Apr 14 03:03:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Deletion of /opt/stack/data/nova/instances/287a236d-d191-465f-bb65-636732ce5689_del complete
Apr 14 03:03:07 user nova-compute[71323]: INFO nova.compute.manager [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 287a236d-d191-465f-bb65-636732ce5689] Took 0.87 seconds to destroy the instance on the hypervisor.
Apr 14 03:03:07 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:03:07 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:08 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] Took 0.74 seconds to deallocate network for instance.
Apr 14 03:03:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-9463dbdf-696c-42ac-9cfe-ab997a85a442 req-57f8fd3f-2365-4f17-b84d-4ac109c032fb service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-deleted-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:03:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.353s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:09 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Deleted allocations for instance 287a236d-d191-465f-bb65-636732ce5689
Apr 14 03:03:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] Acquiring lock "287a236d-d191-465f-bb65-636732ce5689-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] Lock "287a236d-d191-465f-bb65-636732ce5689-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] No waiting events found dispatching network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:09 user nova-compute[71323]: WARNING nova.compute.manager [req-b38513ff-64bf-4c8e-94f3-5df5743212c9 req-8acd3a48-8190-4425-b113-7fed66720611 service nova] [instance: 287a236d-d191-465f-bb65-636732ce5689] Received unexpected event network-vif-plugged-7a4cd76a-7605-4209-9dd2-ec4bced26f20 for instance with vm_state deleted and task_state None.
Apr 14 03:03:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c7fbaf77-f26d-4a19-9d05-00bde0ee120f tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "287a236d-d191-465f-bb65-636732ce5689" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.405s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:22 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441387.7174048, 287a236d-d191-465f-bb65-636732ce5689 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:03:22 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 287a236d-d191-465f-bb65-636732ce5689] VM Stopped (Lifecycle Event)
Apr 14 03:03:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e281192c-e34b-4782-a7fb-90e298cea6b2 None None] [instance: 287a236d-d191-465f-bb65-636732ce5689] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:03:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-changed-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing instance network info cache due to event network-changed-7e8d8948-ff3b-461a-921a-d90f46d9f84c. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:03:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] Acquiring lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:03:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] Acquired lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:03:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Refreshing network info cache for port 7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:03:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updated VIF entry in instance network info cache for port 7e8d8948-ff3b-461a-921a-d90f46d9f84c. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:03:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [{"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.119", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-739a85e7-a861-44da-9a44-21dedf620262 req-08475563-c6ff-4ca5-a14c-091d168e2a18 service nova] Releasing lock "refresh_cache-a78d15ea-7cb2-46e9-af86-8982e053d55a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:36 user nova-compute[71323]: INFO nova.compute.manager [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Terminating instance
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-f60fcdc3-c2bc-479a-84cb-451d1aa131aa req-9f2571cf-c54a-4599-aa8c-b66fa19e3afc service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-55f45afc-99b2-4d8d-9207-824ca1bde700 req-d1e76edf-cb31-4acf-9331-487bf35557d9 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-5b8574e2-f175-4432-a1cb-f17916ddc806 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Instance destroyed successfully.
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.objects.instance [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lazy-loading 'resources' on Instance uuid a78d15ea-7cb2-46e9-af86-8982e053d55a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.119", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "address": "fa:16:3e:e0:84:d8", "network": {"id": "5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-324732614-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.119", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7e8d8948-ff", "ovs_interfaceid": "7e8d8948-ff3b-461a-921a-d90f46d9f84c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7e8d8948-ff, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e0:84:d8,bridge_name='br-int',has_traffic_filtering=True,id=7e8d8948-ff3b-461a-921a-d90f46d9f84c,network=Network(5a4438ff-dedf-4ee8-8ddb-487e4e13ed2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7e8d8948-ff')
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5b8574e2-f1, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:4c:7d,bridge_name='br-int',has_traffic_filtering=True,id=5b8574e2-f175-4432-a1cb-f17916ddc806,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap5b8574e2-f1')
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3e1675a7-2e, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:37 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:70:4f:8a,bridge_name='br-int',has_traffic_filtering=True,id=3e1675a7-2e92-4b45-a657-f7b811975097,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap3e1675a7-2e')
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap04ec9de5-19, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:a6:dd:c9,bridge_name='br-int',has_traffic_filtering=True,id=04ec9de5-1982-4641-8e4f-6eed7f00cf51,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04ec9de5-19')
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfa3f5b14-63, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:bd:ea:26,bridge_name='br-int',has_traffic_filtering=True,id=fa3f5b14-6337-4e66-b990-1ba1cb8ab667,network=Network(5f76fac8-1285-44a2-b867-75852cec358c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa3f5b14-63')
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape99be0b2-4a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:08:0b:91,bridge_name='br-int',has_traffic_filtering=True,id=e99be0b2-4a16-49d7-95ed-51790df3da10,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape99be0b2-4a')
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:01:42Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-555989941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-555989941',id=13,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2LgMvQScP+3OkEhMtxQ8yPEKAnpsqLhU2Ng/RVA17usmzZNC2W9NST4JFW6H9lJuSVw3sE9aw2aZ0d/xexnkTBnPv0G1cmq0AenhUnYQ0O8vu1SCb2h6BoZ3JbQLKp9w==',key_name='tempest-keypair-1725553182',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='de2d10e0d7e7477484e4204841a3980c',ramdisk_id='',reservation_id='r-8re5b6l2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-226254012',owner_user_name='tempest-TaggedBootDevicesTest-226254012-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c3c342144e51496190e32329225ddd2b',uuid=a78d15ea-7cb2-46e9-af86-8982e053d55a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converting VIF {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap65202a6a-2d, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: INFO os_vif [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c0:08:f4,bridge_name='br-int',has_traffic_filtering=True,id=65202a6a-2d5f-420b-a2bb-02ad01831633,network=Network(f8c5c19d-84ab-402e-ab63-9c3249afe5e5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap65202a6a-2d')
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KgfYmwnl', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4565246-9ef0-4785-b6c5-70322bce44df]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: iSNS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: STATIC:
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: FIRMWARE:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:03:38 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf40f9a9-79c1-456d-be94-884cc4ea78a6]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e222bbc8-9362-428e-a485-c66d7d744a80]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cc46ef0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7ea854c-4ca9-4be0-9168-3d32f1cf16c3]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[618d4cda-4af8-4ee9-b9ae-01070d132c2d]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5576b427-fcd6-4376-beef-b0f3cd1260d5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.049s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb233be7-fe95-44cf-b08f-c87603f142ee]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e6731d9-d83b-456e-85ff-0194ca0b2efe]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --logout" returned: 0 in 0.068s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a2c8e63-3529-4a8d-8e47-55466c00c270]: (4, ('Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260]\nLogout of [sid: 11, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260]
Apr 14 03:03:38 user nova-compute[71323]: Logout of [sid: 11, target: iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49, portal: 172.16.0.220,3260] successful.
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:453b7855-8efd-41bd-806c-18ae2c03ac49 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[35dce04f-54cb-4e15-bf57-80b171058980]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.332s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== disconnect_volume: return (332ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VlDJEXwv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7940b6f-100d-4768-914c-dd6ed10663b5]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: iSNS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: STATIC:
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: FIRMWARE:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:03:38 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b10e9108-2c5d-4931-9515-5434cf02d9e4]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3fc8542-567f-4f35-8226-a61594fe2084]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)\ntcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b006253f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-7e8d8948-ff3b-461a-921a-d90f46d9f84c for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-04ec9de5-1982-4641-8e4f-6eed7f00cf51 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-92123f34-488e-45bd-baad-fde5df967be5 req-5b43eaae-f88f-4451-ba4e-118ba26761ac service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b56a944-6cec-4410-99f0-8e55cb2dd367]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b73da6c6-2896-4253-9b2d-960694105869]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4ab8da8-a50a-4aa2-b14a-bbcdf8edb034]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.046s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b61d14e-557b-4e82-8aff-3f8c821dadd8]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[657de295-bbe7-4082-8117-bdadfc3562f1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-5b8574e2-f175-4432-a1cb-f17916ddc806 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-3e1675a7-2e92-4b45-a657-f7b811975097 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-3e1675a7-2e92-4b45-a657-f7b811975097 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-e99be0b2-4a16-49d7-95ed-51790df3da10 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-e99be0b2-4a16-49d7-95ed-51790df3da10 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-unplugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-unplugged-65202a6a-2d5f-420b-a2bb-02ad01831633 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Acquiring lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] No waiting events found dispatching network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING nova.compute.manager [req-26b9c8d5-a8bf-4fd4-87e8-2adc192a0526 req-523ff724-8fb5-418d-a110-e23463eed529 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received unexpected event network-vif-plugged-65202a6a-2d5f-420b-a2bb-02ad01831633 for instance with vm_state active and task_state deleting.
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcf58d9c-b041-456d-b23a-0268869a346a]: (4, ('Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260]\nLogout of [sid: 10, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260]
Apr 14 03:03:38 user nova-compute[71323]: Logout of [sid: 10, target: iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570, portal: 172.16.0.220,3260] successful.
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ba0a016e-f7ab-4aac-8fb3-ecf199678570 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dea30802-eead-4c6e-b44a-6ca442192426]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.325s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== disconnect_volume: return (326ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pGxcVBXE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[97317007-0e1f-44fa-9296-4a70a3e064de]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: iSNS:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]: STATIC:
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:03:38 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:38 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:38 user nova-compute[71323]: FIRMWARE:
Apr 14 03:03:38 user nova-compute[71323]: No targets found.
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:03:38 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:03:38 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21b82e0e-61e3-44bd-97d4-3605c61d6f7f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c2661c3-bfbd-4596-ab86-7bef2466f937]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:38 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b003e3d00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a915d94e-fb49-43e5-9b9b-6b6bbbaaddd4]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e68a26e-822a-4cf7-8b05-53c825182e40]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceacba87-a2f9-43ab-8c48-7be89a77b1a5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6ebd594-419b-452b-8871-4416eae7eb4a]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bea52bfb-6275-41b6-a498-f689802d83f4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[29356866-e43b-4554-aaf5-abcc08c56b24]: (4, ('Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260]\nLogout of [sid: 12, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260]
Apr 14 03:03:39 user nova-compute[71323]: Logout of [sid: 12, target: iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c, portal: 172.16.0.220,3260] successful.
Apr 14 03:03:39 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c11ac81e-755b-4855-85b5-259c815fef2c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9da58e3b-1c66-4534-97c0-80b47cb9afe0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.304s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] <== disconnect_volume: return (305ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Deleting instance files /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a_del
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Deletion of /opt/stack/data/nova/instances/a78d15ea-7cb2-46e9-af86-8982e053d55a_del complete
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.compute.manager [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Took 3.02 seconds to destroy the instance on the hypervisor.
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.compute.manager [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Detaching volume d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.virt.block_device [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Attempting to driver detach volume d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 from mountpoint /dev/vdb
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Attempting to detach device vdb from instance bbb6cd58-ce7a-43ae-876d-51a888193f60 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:03:39 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:03:39 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:03:39 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:03:39 user nova-compute[71323]:   <serial>d1ac2d2b-c18f-4958-a86e-fa298a15c9b8</serial>
Apr 14 03:03:39 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:03:39 user nova-compute[71323]: </disk>
Apr 14 03:03:39 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:03:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdb from instance bbb6cd58-ce7a-43ae-876d-51a888193f60 from the persistent domain config.
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance bbb6cd58-ce7a-43ae-876d-51a888193f60 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:03:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:03:39 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:03:39 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:03:39 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:03:39 user nova-compute[71323]:   <serial>d1ac2d2b-c18f-4958-a86e-fa298a15c9b8</serial>
Apr 14 03:03:39 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:03:39 user nova-compute[71323]: </disk>
Apr 14 03:03:39 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441420.362195, bbb6cd58-ce7a-43ae-876d-51a888193f60 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:03:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdb from instance bbb6cd58-ce7a-43ae-876d-51a888193f60 from the live domain config.
Apr 14 03:03:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mpkibDRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:40 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[74fe6776-b0e4-49ed-b47b-27b052e5fe56]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:03:40 user nova-compute[71323]: No targets found.
Apr 14 03:03:40 user nova-compute[71323]: iSNS:
Apr 14 03:03:40 user nova-compute[71323]: No targets found.
Apr 14 03:03:40 user nova-compute[71323]: STATIC:
Apr 14 03:03:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:03:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:03:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8
Apr 14 03:03:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:03:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:03:40 user nova-compute[71323]: FIRMWARE:
Apr 14 03:03:40 user nova-compute[71323]: No targets found.
Apr 14 03:03:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:03:40 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:03:40 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b3dc814-aabb-43bf-bb05-26fada3e89fe]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee8a2972-24f4-4a41-8b2a-f252cd5af12c]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:40 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:40 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:40 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:40 user nova-compute[71323]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 (non-flash)
Apr 14 03:03:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b003e3250>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0658588-55d9-40ca-b36e-886e1eb20118]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ae51fbe-1bd5-4b9c-806c-5fadca9c9434]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcdfe2f0-d5bb-4f1c-adb8-c6be350be5a8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.046s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6aa661d6-eeb1-4605-833d-cc2eef2e93df]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6f244cc-ed55-4929-8da3-a468cf9f22da]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2b407c4-fb74-4eb9-b9a5-b251d72ed556]: (4, ('Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]\nLogout of [sid: 8, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260]
Apr 14 03:03:40 user nova-compute[71323]: Logout of [sid: 8, target: iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8, portal: 172.16.0.220,3260] successful.
Apr 14 03:03:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1ac2d2b-c18f-4958-a86e-fa298a15c9b8 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6de71349-0e3f-41f5-a2b0-b294f9191fac]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.311s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== disconnect_volume: return (312ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:03:41 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b0001b2f-4592-42ca-a37e-95eb3b6040bb tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.687s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-70505435-ff35-42e7-9252-872ec295097f req-3d091397-9cc6-4952-baa4-c3cd1ae94095 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-deleted-7e8d8948-ff3b-461a-921a-d90f46d9f84c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:41 user nova-compute[71323]: INFO nova.compute.manager [req-70505435-ff35-42e7-9252-872ec295097f req-3d091397-9cc6-4952-baa4-c3cd1ae94095 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Neutron deleted interface 7e8d8948-ff3b-461a-921a-d90f46d9f84c; detaching it from the instance and deleting it from the info cache
Apr 14 03:03:41 user nova-compute[71323]: DEBUG nova.network.neutron [req-70505435-ff35-42e7-9252-872ec295097f req-3d091397-9cc6-4952-baa4-c3cd1ae94095 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [{"id": "5b8574e2-f175-4432-a1cb-f17916ddc806", "address": "fa:16:3e:b1:4c:7d", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.150", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5b8574e2-f1", "ovs_interfaceid": "5b8574e2-f175-4432-a1cb-f17916ddc806", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "3e1675a7-2e92-4b45-a657-f7b811975097", "address": "fa:16:3e:70:4f:8a", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.34", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3e1675a7-2e", "ovs_interfaceid": "3e1675a7-2e92-4b45-a657-f7b811975097", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "address": "fa:16:3e:a6:dd:c9", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.132", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04ec9de5-19", "ovs_interfaceid": "04ec9de5-1982-4641-8e4f-6eed7f00cf51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "address": "fa:16:3e:bd:ea:26", "network": {"id": "5f76fac8-1285-44a2-b867-75852cec358c", "bridge": "br-int", "label": "tempest-device-tagging-net1-701376356", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.78", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa3f5b14-63", "ovs_interfaceid": "fa3f5b14-6337-4e66-b990-1ba1cb8ab667", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "e99be0b2-4a16-49d7-95ed-51790df3da10", "address": "fa:16:3e:08:0b:91", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape99be0b2-4a", "ovs_interfaceid": "e99be0b2-4a16-49d7-95ed-51790df3da10", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "65202a6a-2d5f-420b-a2bb-02ad01831633", "address": "fa:16:3e:c0:08:f4", "network": {"id": "f8c5c19d-84ab-402e-ab63-9c3249afe5e5", "bridge": "br-int", "label": "tempest-device-tagging-net2-109827133", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "de2d10e0d7e7477484e4204841a3980c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap65202a6a-2d", "ovs_interfaceid": "65202a6a-2d5f-420b-a2bb-02ad01831633", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-70505435-ff35-42e7-9252-872ec295097f req-3d091397-9cc6-4952-baa4-c3cd1ae94095 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Detach interface failed, port_id=7e8d8948-ff3b-461a-921a-d90f46d9f84c, reason: Instance a78d15ea-7cb2-46e9-af86-8982e053d55a could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=71323) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3323}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.objects.instance [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:42 user nova-compute[71323]: WARNING nova.compute.manager [req-01a78ce5-dc66-4521-8132-993366d5af09 req-494159dd-22fa-4be5-9bfe-b1cfd26ebd18 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state powering-off.
Apr 14 03:03:42 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance destroyed successfully.
Apr 14 03:03:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:03:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1ea06b1e-7aef-4788-b220-323c6edba48e tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.858s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:43 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Took 4.02 seconds to deallocate network for instance.
Apr 14 03:03:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-a7f3ae17-94f2-44ac-b9ca-9c728d06dbfe req-ced40d0e-8db3-4eca-966b-c3b1770b92e7 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-deleted-65202a6a-2d5f-420b-a2bb-02ad01831633 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-a7f3ae17-94f2-44ac-b9ca-9c728d06dbfe req-ced40d0e-8db3-4eca-966b-c3b1770b92e7 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-deleted-e99be0b2-4a16-49d7-95ed-51790df3da10 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-a7f3ae17-94f2-44ac-b9ca-9c728d06dbfe req-ced40d0e-8db3-4eca-966b-c3b1770b92e7 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-deleted-fa3f5b14-6337-4e66-b990-1ba1cb8ab667 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-a7f3ae17-94f2-44ac-b9ca-9c728d06dbfe req-ced40d0e-8db3-4eca-966b-c3b1770b92e7 service nova] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Received event network-vif-deleted-04ec9de5-1982-4641-8e4f-6eed7f00cf51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:03:44 user nova-compute[71323]: WARNING nova.compute.manager [req-1b13e9e8-3c38-4e2b-83a0-1db6c8f1304f req-ce90c1c8-2c53-476a-b457-197ada21ad13 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state stopped and task_state None.
Apr 14 03:03:45 user nova-compute[71323]: INFO nova.compute.manager [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Took 2.39 seconds to detach 3 volumes for instance.
Apr 14 03:03:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:45 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:03:45 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:03:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.737s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:46 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Deleted allocations for instance a78d15ea-7cb2-46e9-af86-8982e053d55a
Apr 14 03:03:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ca8f998b-c574-4c82-9b90-52772525f81c tempest-TaggedBootDevicesTest-226254012 tempest-TaggedBootDevicesTest-226254012-project-member] Lock "a78d15ea-7cb2-46e9-af86-8982e053d55a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 10.359s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:03:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-changed-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Refreshing instance network info cache due to event network-changed-b975970e-4ce0-4b1c-b1b3-956befa892e0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] Acquiring lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] Acquired lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG nova.network.neutron [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Refreshing network info cache for port b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'flavor' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.095s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:48 user nova-compute[71323]: INFO nova.compute.manager [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attaching volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f to /dev/vdb
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.utils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b86366b6-ef05-483c-8300-8adc8847f58f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG nova.network.neutron [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updated VIF entry in instance network info cache for port b975970e-4ce0-4b1c-b1b3-956befa892e0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG nova.network.neutron [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updating instance_info_cache with network_info: [{"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.49", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d413425a-7352-41d0-b014-b9dab5d149f9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-174cd578-2ca3-41d3-81ac-e4332a3f4814 req-9d02ea7e-e8df-4b0c-8cb0-ad319ef01ceb service nova] Releasing lock "refresh_cache-b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bde2082d-58fc-41e1-b6b4-182d20e4a29d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1566d2a5-873e-4c71-ac06-fe78efb016b0]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:03:48 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5b6577c-3315-45fb-9c45-e637ab3998b9]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG os_brick.utils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updating existing volume attachment record: e2b6388f-42f4-4160-ba5f-2306922c9f5a {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:48 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:03:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:52 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:52 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:52 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b62d982-67e4-4b0d-8fa1-5e9efc784d89]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:03:52 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[84acb878-e95b-49bd-a533-3028586d858a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f] added
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9465a19d-8276-4f9e-870b-4b5fce7e15b1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[76cb4529-a4dd-423b-9cf8-567f64689a71]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DtCpadIK {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DtCpadIK" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccf0c859-b673-4c95-858b-dab8e70c64c1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DtCpadIK'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1c8b38c-933c-440a-916d-a3ddc0f75e9f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef5da695-d703-4ec6-a475-4630144aa72c]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441417.7815576, a78d15ea-7cb2-46e9-af86-8982e053d55a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:03:52 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] VM Stopped (Lifecycle Event)
Apr 14 03:03:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ca4c3eed-0892-4cab-9c03-2582d22026b7 None None] [instance: a78d15ea-7cb2-46e9-af86-8982e053d55a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --login" returned: 0 in 0.055s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6ec285c-083c-4adb-86f8-05606d8e5036]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]
Apr 14 03:03:52 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b74a463-2108-48a0-9f78-6a9251fbd65e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[58564291-8193-45de-8295-bdf39037e31a]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:03:52 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:03:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.291s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] HCTL ('33', '-', '-', 0) found on session 15 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[95cadf91-72d0-4b4c-b5cd-fcd305fc81de]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Searching for a device in session 15 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:03:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.321s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] <== connect_volume: return (1323ms) {'type': 'block', 'scsi_wwn': '26132653533633730', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26132653533633730', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'flavor' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:03:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:03:53 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:03:53 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:03:53 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:03:53 user nova-compute[71323]:   <serial>0032d61d-2851-4bde-b7cd-42043f3b1a7f</serial>
Apr 14 03:03:53 user nova-compute[71323]: </disk>
Apr 14 03:03:53 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] No VIF found with MAC fa:16:3e:2b:e1:ca, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b67cc212-2026-4f04-8ae8-c311818e9eb9 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.366s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:03:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.utils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e04306d-4582-4006-b3a5-a9e1cd66d2f9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28c1826a-b5c4-483f-b5a8-6827b1d332e2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb2973b5-f410-4ade-8890-52587c549156]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[156a5a3d-2bed-4a39-aeec-e33d6f2b13a9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:03:55 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b06d00dd-fad9-404b-b4de-708c570b2cd7]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG os_brick.utils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:03:55 user nova-compute[71323]: INFO nova.compute.manager [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Swapping volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f for 80d66b06-08ad-4e2d-b0b7-5906492cd660
Apr 14 03:03:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating instance_info_cache with network_info: [{"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:03:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441422.8962576, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:03:57 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Stopped (Lifecycle Event)
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-30b077f2-67f8-45d2-9368-a352448c3271 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:03:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-30b077f2-67f8-45d2-9368-a352448c3271 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: stopped, current task_state: None, current DB power_state: 4, VM power_state: 4 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:03:59 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:03:59 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7696MB free_disk=26.52780532836914GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f970adb1-fe1b-4b58-83aa-45207a6211ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance dc8c2e40-f896-4d21-9927-67330485d77d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660', 'serial': '80d66b06-08ad-4e2d-b0b7-5906492cd660'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'serial'} {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:03:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:03:59 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b34967ec-8353-42b9-9c18-4a1ea1faa4de]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:03:59 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6659c3c9-5a52-4a80-91b7-a76266eacbf7]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660] added
Apr 14 03:03:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efbc8e7b-1c8e-4cc6-b08d-95e77382cbc3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:03:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.062s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[db964132-6a29-4c2d-a81d-afaa7006658a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FGovYJtj {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FGovYJtj" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3db905b4-d35e-412f-8096-7e3f167972da]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FGovYJtj'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bc64dca-fe89-4473-8525-2269150dfea9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a433665-2661-464a-b0ec-eab563b13df8]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.652s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --login" returned: 0 in 0.069s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[52a0643d-03bc-4ba4-85cd-122b8b7316a8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260]
Apr 14 03:04:00 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28d82389-91a0-46b1-af75-e6d0c85175f1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2c72028-e26c-4430-b6d5-3ddfc4947af8]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:00 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.384s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] HCTL ('34', '-', '-', 0) found on session 16 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6492b5fd-c880-45a7-b60d-807b44ce99fc]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Searching for a device in session 16 and hctl ['34', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid f970adb1-fe1b-4b58-83aa-45207a6211ef {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.088s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.088s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.091s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.108s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.103s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.110s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.107s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.112s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.111s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: INFO nova.compute.manager [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Terminating instance
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-unplugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] No waiting events found dispatching network-vif-unplugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-1ef518eb-3bbc-47f7-9689-28e14a245828 req-1a71a0d4-9820-42a9-bde6-f703c4edf317 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-unplugged-9a41964d-ca86-41d7-8287-1af6e94403b4 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Searching for a device in session 16 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.414s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== connect_volume: return (1416ms) {'type': 'block', 'scsi_wwn': '23833346630373930', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23833346630373930', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lazy-loading 'flavor' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Instance destroyed successfully.
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lazy-loading 'resources' on Instance uuid 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:58:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1530771040',display_name='tempest-VolumesAdminNegativeTest-server-1530771040',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1530771040',id=7,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLRjkf56mNaAsNKFgnUHc6YtgNOI11oCC45IpAljzRBHWdlhRZJtff4EnryoIR2UUdedBXj1eNEYfP5CFJHsIuaLL73yA24OQjZOf+7GA5PGlabqvgSGMvLYjqcMcQgnmA==',key_name='tempest-keypair-309889138',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='94a1eb61159b43228db0d529a5be816d',ramdisk_id='',reservation_id='r-klkjfltg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-1232189050',owner_user_name='tempest-VolumesAdminNegativeTest-1232189050-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:58:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f0d4916eb93c4b3ebbbd2af4b0cdacc1',uuid=3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converting VIF {"id": "9a41964d-ca86-41d7-8287-1af6e94403b4", "address": "fa:16:3e:7a:2f:92", "network": {"id": "2b0bbe21-b516-4656-8326-a02e9d0e0d38", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1516903575-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.85", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "94a1eb61159b43228db0d529a5be816d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9a41964d-ca", "ovs_interfaceid": "9a41964d-ca86-41d7-8287-1af6e94403b4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG os_vif [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9a41964d-ca, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:01 user nova-compute[71323]: INFO os_vif [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7a:2f:92,bridge_name='br-int',has_traffic_filtering=True,id=9a41964d-ca86-41d7-8287-1af6e94403b4,network=Network(2b0bbe21-b516-4656-8326-a02e9d0e0d38),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9a41964d-ca')
Apr 14 03:04:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Deleting instance files /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5_del
Apr 14 03:04:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Deletion of /opt/stack/data/nova/instances/3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5_del complete
Apr 14 03:04:01 user nova-compute[71323]: INFO nova.compute.manager [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 14 03:04:01 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:04:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 0 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:04:02 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Took 0.93 seconds to deallocate network for instance.
Apr 14 03:04:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 98566144 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.396s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 210763776 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:02 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Deleted allocations for instance 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5
Apr 14 03:04:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] Acquiring lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] No waiting events found dispatching network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:03 user nova-compute[71323]: WARNING nova.compute.manager [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received unexpected event network-vif-plugged-9a41964d-ca86-41d7-8287-1af6e94403b4 for instance with vm_state deleted and task_state None.
Apr 14 03:04:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-336fcf56-2cef-4204-a51b-0276fb3dad37 req-dba3ea7b-9042-4d42-b864-78180c5a50c8 service nova] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Received event network-vif-deleted-9a41964d-ca86-41d7-8287-1af6e94403b4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-194eb9be-e29b-409c-b59d-7c63e8001d95 tempest-VolumesAdminNegativeTest-1232189050 tempest-VolumesAdminNegativeTest-1232189050-project-member] Lock "3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.492s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 407896064 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 594542592 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 787480576 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 932184064 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a46fa0cc-a0b2-41fe-a29a-3fde1498bd96]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:04:05 user nova-compute[71323]: No targets found.
Apr 14 03:04:05 user nova-compute[71323]: iSNS:
Apr 14 03:04:05 user nova-compute[71323]: No targets found.
Apr 14 03:04:05 user nova-compute[71323]: STATIC:
Apr 14 03:04:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:04:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f
Apr 14 03:04:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660
Apr 14 03:04:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:05 user nova-compute[71323]: FIRMWARE:
Apr 14 03:04:05 user nova-compute[71323]: No targets found.
Apr 14 03:04:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:04:05 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:04:05 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[02395145-c944-496d-b6fe-32ac31d37c15]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21f14e16-3819-4ef3-a9fc-9520d72ba22a]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:05 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd5b40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fc6ab22-b72d-4644-b681-610c068c32e2]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de431a63-1a37-499c-9bd2-e541b28506cc]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaa68a2d-2e64-41bb-97b3-a463c5194cca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.039s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b11ee161-6a14-4c73-aba9-7e6cf8ee97d8]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d43861a-e1b6-497a-a35b-0d7355c78f07]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --logout" returned: 0 in 0.088s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[33a8e65e-ee1b-4a0a-b2b9-5f11549c5c5a]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]
Apr 14 03:04:05 user nova-compute[71323]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[50bb66c1-6710-4e19-a665-cdb2aba5dd4e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.322s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== disconnect_volume: return (323ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660', 'serial'} {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: removing Cinder connection for volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: Updating volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "FGovYJtj", "auth_password": "Gl8HmxdDhXEIVU", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdc", "volume_id": "80d66b06-08ad-4e2d-b0b7-5906492cd660"}, "status": "reserved", "instance": "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49", "attached_at": "", "detached_at": "", "volume_id": "80d66b06-08ad-4e2d-b0b7-5906492cd660", "serial": "80d66b06-08ad-4e2d-b0b7-5906492cd660"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660', 'no_device': None, 'attachment_id': 'ca0a896b-9c6c-43b7-82bc-710261959b12'} {{(pid=71323) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-88a3c668-9f74-4a4d-8655-a29086dfa3cd tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 11.760s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 6.358s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.029s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.utils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f3160b8-baa5-45b8-a75d-aeaf0c95fa16]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[97f1c770-a632-45af-912c-e5cb3a1c25a7]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[16076b0e-e9f8-4fd7-a6ef-2c6a6170a45c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0274a10-fe69-42a0-baec-c56c517672ec]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:07 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:04:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a768784e-451c-42b3-81c0-fd817529c2e2]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:04:07 user nova-compute[71323]: DEBUG os_brick.utils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== get_connector_properties: return (110ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:07 user nova-compute[71323]: INFO nova.compute.manager [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Swapping volume 80d66b06-08ad-4e2d-b0b7-5906492cd660 for 0032d61d-2851-4bde-b7cd-42043f3b1a7f
Apr 14 03:04:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'serial': '0032d61d-2851-4bde-b7cd-42043f3b1a7f'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc', 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660'}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660', 'serial'} {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7656}}
Apr 14 03:04:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:04:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:11 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:11 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:12 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93b3aac3-4aee-4417-a85a-745657ef3ca4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:04:12 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6675903-43ea-432b-a0ba-d8290a2257a7]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f] added
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[48513b24-477d-4b86-9356-58f5bc758f59]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[45fdb164-277a-4b41-a167-384c49c900f2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DtCpadIK {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DtCpadIK" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d73545f-4c43-4b2d-ac0b-00ce2beccdc0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DtCpadIK'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cdcde32-f43a-4b5d-97c2-431746be62b8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[51996a67-3729-4df4-ab25-b471824bf54b]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[60c79471-3e5c-4d4f-ba01-aea006358e53]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]
Apr 14 03:04:12 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1377d14-0580-4e8a-b938-44d450a8d84e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[59fb8540-b954-4712-a161-001511100974]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:12 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:12 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.287s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] HCTL ('33', '-', '-', 0) found on session 17 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[736faaa3-9402-4ec2-abfc-32558ba79381]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Searching for a device in session 17 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:04:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.323s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== connect_volume: return (1324ms) {'type': 'block', 'scsi_wwn': '26132653533633730', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26132653533633730', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:04:13 user nova-compute[71323]: DEBUG nova.objects.instance [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lazy-loading 'flavor' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 6291456 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 202375168 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 356515840 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 562036736 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 722468864 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 902823936 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441441.3797815, 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:04:16 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] VM Stopped (Lifecycle Event)
Apr 14 03:04:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21578664-b11d-46bb-876b-bc6b84d7685d None None] [instance: 3ae6f41c-d3f6-4c45-9317-64dc9c7a98a5] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 1069547520 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FGovYJtj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc', 'volume_id': '80d66b06-08ad-4e2d-b0b7-5906492cd660': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:17 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3dc250b-4400-4343-b9e9-1d04da8a1105]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:04:17 user nova-compute[71323]: No targets found.
Apr 14 03:04:17 user nova-compute[71323]: iSNS:
Apr 14 03:04:17 user nova-compute[71323]: No targets found.
Apr 14 03:04:17 user nova-compute[71323]: STATIC:
Apr 14 03:04:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:04:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f
Apr 14 03:04:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660
Apr 14 03:04:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:17 user nova-compute[71323]: FIRMWARE:
Apr 14 03:04:17 user nova-compute[71323]: No targets found.
Apr 14 03:04:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:04:17 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:04:17 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[043f5d4e-42f8-47d6-a9a8-993bdecfc037]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7b56e77-5ece-4437-b46f-8b3d9ae244d0]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 (non-flash)
Apr 14 03:04:17 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cdcb2e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[48a07a6d-322d-482f-820e-b9d1280892da]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8779dfea-e876-440a-ac0b-c43b77e8535f]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18869bc4-3a86-456f-8d3b-a14d0f78b78b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.051s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[83af3443-1f92-4eb3-b976-5ecb83e42fce]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[44bda6ed-a219-411c-ba94-3a5af3f83f12]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --logout" returned: 0 in 0.081s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5cd7b52-f806-4782-82ad-58ac5ee052b2]: (4, ('Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260]\nLogout of [sid: 16, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260]
Apr 14 03:04:17 user nova-compute[71323]: Logout of [sid: 16, target: iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:80d66b06-08ad-4e2d-b0b7-5906492cd660 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea40de50-b7b9-4d8f-8a9c-523b07d550a4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.323s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] <== disconnect_volume: return (325ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': 'b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49', 'attached_at': '', 'detached_at': '', 'volume_id': '0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'serial'} {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7665}}
Apr 14 03:04:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: removing Cinder connection for volume 80d66b06-08ad-4e2d-b0b7-5906492cd660 {{(pid=71323) _swap_volume /opt/stack/nova/nova/compute/manager.py:7710}}
Apr 14 03:04:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] swap_volume: Updating volume 80d66b06-08ad-4e2d-b0b7-5906492cd660 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "DtCpadIK", "auth_password": "eUsmbH1Qj5ZP0o", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdb", "volume_id": "0032d61d-2851-4bde-b7cd-42043f3b1a7f"}, "status": "reserved", "instance": "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49", "attached_at": "", "detached_at": "", "volume_id": "0032d61d-2851-4bde-b7cd-42043f3b1a7f", "serial": "0032d61d-2851-4bde-b7cd-42043f3b1a7f"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'no_device': None, 'attachment_id': '5b96afaa-298b-4258-8065-e9ba3e31a4f4'} {{(pid=71323) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7858}}
Apr 14 03:04:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-52fc5624-5b30-4fef-bd2d-fff64eca21ec tempest-TestVolumeSwap-1976683534 tempest-TestVolumeSwap-1976683534-project-admin] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 11.286s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:19 user nova-compute[71323]: INFO nova.compute.manager [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Detaching volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f
Apr 14 03:04:19 user nova-compute[71323]: INFO nova.virt.block_device [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Attempting to driver detach volume 0032d61d-2851-4bde-b7cd-42043f3b1a7f from mountpoint /dev/vdb
Apr 14 03:04:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Attempting to detach device vdb from instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:04:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:19 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:19 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:04:19 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:19 user nova-compute[71323]:   <serial>0032d61d-2851-4bde-b7cd-42043f3b1a7f</serial>
Apr 14 03:04:19 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:04:19 user nova-compute[71323]: </disk>
Apr 14 03:04:19 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:19 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Successfully detached device vdb from instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 from the persistent domain config.
Apr 14 03:04:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:04:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:19 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:19 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:04:19 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:19 user nova-compute[71323]:   <serial>0032d61d-2851-4bde-b7cd-42043f3b1a7f</serial>
Apr 14 03:04:19 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:04:19 user nova-compute[71323]: </disk>
Apr 14 03:04:19 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441460.197297, b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:04:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Successfully detached device vdb from instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 from the live domain config.
Apr 14 03:04:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DtCpadIK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb', 'volume_id': '0032d61d-2851-4bde-b7cd-42043f3b1a7f': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:20 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ceb4dc3-6cd3-429e-82fd-e8a679e9ebb6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:04:20 user nova-compute[71323]: No targets found.
Apr 14 03:04:20 user nova-compute[71323]: iSNS:
Apr 14 03:04:20 user nova-compute[71323]: No targets found.
Apr 14 03:04:20 user nova-compute[71323]: STATIC:
Apr 14 03:04:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:04:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f
Apr 14 03:04:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:20 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:20 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:20 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:20 user nova-compute[71323]: FIRMWARE:
Apr 14 03:04:20 user nova-compute[71323]: No targets found.
Apr 14 03:04:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:04:20 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:04:20 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a1c36f2-9e8e-4b3b-a558-b5bd1e7becbc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5aee95a-0fb7-43cb-8e37-d8788ac995ea]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:20 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:20 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:20 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:20 user nova-compute[71323]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f (non-flash)
Apr 14 03:04:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b0014b2e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[91317b02-db41-4ec3-b06a-94b459992954]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c46cbf2-3c63-4a8e-b7a2-42d7ba7ef650]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[df8038af-2e67-44ce-8b3d-d214b86ad4c1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.036s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a72365e-24f4-4518-9570-4edf4d56e6cc]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[29b2d1f9-e862-448f-80f3-0e20342a8e6e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --logout" returned: 0 in 0.051s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6c2150c-4614-43a8-8a29-8fac87ce98c0]: (4, ('Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]\nLogout of [sid: 17, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260]
Apr 14 03:04:20 user nova-compute[71323]: Logout of [sid: 17, target: iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:20 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0032d61d-2851-4bde-b7cd-42043f3b1a7f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5db65ea4-2617-438d-9773-0d5f8bcd5349]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.295s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] <== disconnect_volume: return (296ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:04:21 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'flavor' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a0f0a80d-f60f-4875-96f0-6f7b68070488 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.650s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:04:24 user nova-compute[71323]: INFO nova.compute.claims [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Claim successful on node user
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.428s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:04:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:04:24 user nova-compute[71323]: INFO nova.virt.block_device [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Booting with volume snapshot f78b3968-f436-44d2-8fb3-4f3d899eeed5 at /dev/vda
Apr 14 03:04:24 user nova-compute[71323]: DEBUG nova.policy [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing instance network info cache due to event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] Acquiring lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] Acquired lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing network info cache for port a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:25 user nova-compute[71323]: INFO nova.compute.manager [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Attaching volume 95596aa7-5fd3-48b5-a091-0379a2e77fcc to /dev/vdb
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.utils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f01d3f7c-0f09-4291-9396-9ece645d20e5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5defec37-733f-462b-9f24-9cf9ab49a647]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb0ccaba-50eb-46a4-8f17-97061313da06]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2a9e404-ef18-4d5b-a73b-a7edd2fec6f2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:25 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fb3e388-5b4f-4b8d-924a-4809b5694523]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG os_brick.utils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating existing volume attachment record: 92d031d7-202c-4eeb-b2cb-9e7ee2abc59f {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Successfully created port: a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updated VIF entry in instance network info cache for port a36f717a-9a8b-43c9-ba01-0f03a522d166. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating instance_info_cache with network_info: [{"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.7", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:04:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6b74c45-9f87-4c14-beb0-a19b6da85496 req-892a22f2-501c-4ef9-843e-6d74aa5b77d5 service nova] Releasing lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Successfully updated port: a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-changed-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Refreshing instance network info cache due to event network-changed-a07f6861-f9f6-42c0-8d36-07db58ad1cb2. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] Acquiring lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Updating instance_info_cache with network_info: [{"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Instance network_info: |[{"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] Acquired lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:04:26 user nova-compute[71323]: DEBUG nova.network.neutron [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Refreshing network info cache for port a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:04:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Updated VIF entry in instance network info cache for port a07f6861-f9f6-42c0-8d36-07db58ad1cb2. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:04:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Updating instance_info_cache with network_info: [{"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:04:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c391dbb9-5725-4e6e-b0dc-e8458954ffab req-594ec0ec-7720-497e-9482-6b5d68c98d92 service nova] Releasing lock "refresh_cache-62cad020-0344-4e4e-add9-e5bc6175adbb" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:04:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pTqrRZGK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:30 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:30 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:30 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[65980396-6f1a-4d33-a470-30496b43189c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:04:30 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d154bee-357b-40d8-877a-7463750bbfb1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc] added
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[50533830-1224-43f3-9be2-7b76c3f416f1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.037s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e0f01ed-7bd7-4597-bd7f-91377668eb2b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pTqrRZGK {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pTqrRZGK" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[91de8e3c-4225-45b1-9edf-adf1fca360cc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pTqrRZGK'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9202da4-ad5c-41ed-9a99-8715da5e60d7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b2f2ab8-f6d7-4a07-9f98-3da95de954ca]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[131cfb0d-3bf7-4279-b61c-0c9c652f9aa8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260]
Apr 14 03:04:30 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a14e9ec6-945f-4451-88b8-9c82b7522b58]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[02ce60fd-8183-488a-a4fc-925effd97320]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:30 user nova-compute[71323]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)
Apr 14 03:04:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.317s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] HCTL ('33', '-', '-', 0) found on session 18 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8620794-d43b-4c9b-89f6-5ab618c994e9]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Searching for a device in session 18 and hctl ['33', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.utils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a131b03-0a07-4343-bfd6-af9dd007ed69]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[146e0814-51c3-45fa-b4bf-20548dda62eb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7117200d-7627-4f57-adf1-774687756bb9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c11aa1c-cb67-4d67-ae13-5c6ed20cae22]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:04:30 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:04:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[75595194-11b2-4050-9353-354a6e2b12a6]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG os_brick.utils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (110ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:30 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Updating existing volume attachment record: 45b15196-b557-451c-9c76-a4a817032cbc {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Searching for a device in session 18 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pTqrRZGK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.358s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== connect_volume: return (1360ms) {'type': 'block', 'scsi_wwn': '23063386564356561', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23063386564356561', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:04:31 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:31 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:04:31 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:31 user nova-compute[71323]:   <serial>95596aa7-5fd3-48b5-a091-0379a2e77fcc</serial>
Apr 14 03:04:31 user nova-compute[71323]: </disk>
Apr 14 03:04:31 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:04:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:04:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:04:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] No VIF found with MAC fa:16:3e:ab:25:10, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:04:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bb713ff7-f256-4c55-8653-7c4b2b16a2ad tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.733s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:33 user nova-compute[71323]: INFO nova.compute.manager [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Unrescuing
Apr 14 03:04:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:04:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:04:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:04:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:33 user nova-compute[71323]: INFO nova.compute.manager [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Detaching volume 95596aa7-5fd3-48b5-a091-0379a2e77fcc
Apr 14 03:04:34 user nova-compute[71323]: INFO nova.virt.block_device [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Attempting to driver detach volume 95596aa7-5fd3-48b5-a091-0379a2e77fcc from mountpoint /dev/vdb
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Attempting to detach device vdb from instance dc8c2e40-f896-4d21-9927-67330485d77d from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:34 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:34 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:04:34 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:34 user nova-compute[71323]:   <serial>95596aa7-5fd3-48b5-a091-0379a2e77fcc</serial>
Apr 14 03:04:34 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:04:34 user nova-compute[71323]: </disk>
Apr 14 03:04:34 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:34 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully detached device vdb from instance dc8c2e40-f896-4d21-9927-67330485d77d from the persistent domain config.
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance dc8c2e40-f896-4d21-9927-67330485d77d from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:34 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:34 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:04:34 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:34 user nova-compute[71323]:   <serial>95596aa7-5fd3-48b5-a091-0379a2e77fcc</serial>
Apr 14 03:04:34 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:04:34 user nova-compute[71323]: </disk>
Apr 14 03:04:34 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441474.619939, dc8c2e40-f896-4d21-9927-67330485d77d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:04:34 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully detached device vdb from instance dc8c2e40-f896-4d21-9927-67330485d77d from the live domain config.
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pTqrRZGK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:34 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b8405c2-f889-41bb-820c-431d150b147a]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:04:34 user nova-compute[71323]: No targets found.
Apr 14 03:04:34 user nova-compute[71323]: iSNS:
Apr 14 03:04:34 user nova-compute[71323]: No targets found.
Apr 14 03:04:34 user nova-compute[71323]: STATIC:
Apr 14 03:04:34 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:04:34 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:34 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:34 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:34 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:34 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:34 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc
Apr 14 03:04:34 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:34 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:34 user nova-compute[71323]: FIRMWARE:
Apr 14 03:04:34 user nova-compute[71323]: No targets found.
Apr 14 03:04:34 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:04:34 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:04:34 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[03cde8d9-244c-48d1-8ec1-b382f7dca866]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d9a3558-3014-4bad-a0e6-424508bbfd05]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:34 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:34 user nova-compute[71323]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)
Apr 14 03:04:34 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:34 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:34 user nova-compute[71323]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc (non-flash)
Apr 14 03:04:34 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00148670>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.network.neutron [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[29d87aea-91a1-4ab6-bb5a-f334c5278fbc]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG nova.objects.instance [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'flavor' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[11d72c74-ed56-4fab-bdb3-8dee114e1da4]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[91f18a9f-454a-4e28-8013-bf638beb8b88]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.070s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd2a56b-b4ca-43a4-b743-0edf472b352a]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c275c52b-51db-45a8-838d-7c2a72c27d2c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --logout" returned: 0 in 0.074s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6915eb73-4514-461c-ae0a-0cd9cae771d0]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260]
Apr 14 03:04:35 user nova-compute[71323]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:35 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:95596aa7-5fd3-48b5-a091-0379a2e77fcc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3218e6ba-52ff-49c9-a6ad-0b2005b42de6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.406s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] <== disconnect_volume: return (408ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:35 user nova-compute[71323]: WARNING nova.compute.manager [req-194af282-d87a-483a-98a0-d0452ece7b4c req-809f0fd6-d6b4-420c-b6fb-401d08744a0a service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:04:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance destroyed successfully.
Apr 14 03:04:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'flavor' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70ceee7d-ed81-4567-995b-240deed754da tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.136s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:04:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Creating image(s)
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Ensure instance console log exists: /opt/stack/data/nova/instances/62cad020-0344-4e4e-add9-e5bc6175adbb/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Start _get_guest_xml network_info=[{"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-14T03:04:22Z,direct_url=<?>,disk_format='qcow2',id=b58349d2-f4ea-42ed-81b9-5113fd237947,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-559111257',owner='2d2e49e8e92e416faebf55e3d4e0e087',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-14T03:04:23Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '45b15196-b557-451c-9c76-a4a817032cbc', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ImLoQRXC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '62cad020-0344-4e4e-add9-e5bc6175adbb', 'attached_at': '', 'detached_at': '', 'volume_id': '423d87a6-a274-408c-ab7c-e91e4edfd963', 'serial': '423d87a6-a274-408c-ab7c-e91e4edfd963'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:04:36 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:04:36 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-04-14T03:04:22Z,direct_url=<?>,disk_format='qcow2',id=b58349d2-f4ea-42ed-81b9-5113fd237947,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-559111257',owner='2d2e49e8e92e416faebf55e3d4e0e087',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-04-14T03:04:23Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ImLoQRXC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:36 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9aeca620-3da2-4022-bf2d-ce88d048c3ab]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:04:36 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b443ca61-24b2-475d-876e-ee3b5aa30483]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963] added
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[922aef1e-da86-4284-ba98-25df990472c3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[773a90b6-5dcb-474f-af5d-b38742c01c68]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ImLoQRXC {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ImLoQRXC" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e68282d1-bf2d-47c0-b8a1-35acedc50f2d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ImLoQRXC'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[98bb1974-39d8-42aa-a51e-1dc5b30d06f1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e5d4ce5-96f2-427c-a6a9-71efee7a1096]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --login" returned: 0 in 0.056s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[da89ba6b-6ee0-486e-8492-0e80cb65052f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260]
Apr 14 03:04:36 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[be1cafb4-9dcd-47cb-a44f-63fde0c2c37a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[87138eb0-3a76-4ddb-b316-4fdee7b7154e]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:36 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:04:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.274s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('33', '-', '-', 0) found on session 19 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceb495b7-31a3-4657-9007-c57e29286d1d]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 19 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ImLoQRXC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:04:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.308s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1309ms) {'type': 'block', 'scsi_wwn': '23636653734346339', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23636653734346339', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:37 user nova-compute[71323]: WARNING nova.compute.manager [req-442d37f2-8598-45fc-9415-9b0b5a4ef21d req-1cddba32-e230-44b1-954a-ca5c9b141938 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-380447828',id=17,image_ref='b58349d2-f4ea-42ed-81b9-5113fd237947',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-nv1ajnpj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-910285030',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:04:25Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=62cad020-0344-4e4e-add9-e5bc6175adbb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.objects.instance [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid 62cad020-0344-4e4e-add9-e5bc6175adbb {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:04:37 user nova-compute[71323]:   <uuid>62cad020-0344-4e4e-add9-e5bc6175adbb</uuid>
Apr 14 03:04:37 user nova-compute[71323]:   <name>instance-00000011</name>
Apr 14 03:04:37 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:04:37 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:04:37 user nova-compute[71323]:   <metadata>
Apr 14 03:04:37 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:04:37 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-image-snapshot-server-380447828</nova:name>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:04:36</nova:creationTime>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:04:37 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:04:37 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:owner>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 03:04:37 user nova-compute[71323]:       </nova:owner>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:root type="image" uuid="b58349d2-f4ea-42ed-81b9-5113fd237947"/>
Apr 14 03:04:37 user nova-compute[71323]:       <nova:ports>
Apr 14 03:04:37 user nova-compute[71323]:         <nova:port uuid="a07f6861-f9f6-42c0-8d36-07db58ad1cb2">
Apr 14 03:04:37 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:04:37 user nova-compute[71323]:         </nova:port>
Apr 14 03:04:37 user nova-compute[71323]:       </nova:ports>
Apr 14 03:04:37 user nova-compute[71323]:     </nova:instance>
Apr 14 03:04:37 user nova-compute[71323]:   </metadata>
Apr 14 03:04:37 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:04:37 user nova-compute[71323]:     <system>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="serial">62cad020-0344-4e4e-add9-e5bc6175adbb</entry>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="uuid">62cad020-0344-4e4e-add9-e5bc6175adbb</entry>
Apr 14 03:04:37 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:04:37 user nova-compute[71323]:     </system>
Apr 14 03:04:37 user nova-compute[71323]:   </sysinfo>
Apr 14 03:04:37 user nova-compute[71323]:   <os>
Apr 14 03:04:37 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:04:37 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:04:37 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:04:37 user nova-compute[71323]:   </os>
Apr 14 03:04:37 user nova-compute[71323]:   <features>
Apr 14 03:04:37 user nova-compute[71323]:     <acpi/>
Apr 14 03:04:37 user nova-compute[71323]:     <apic/>
Apr 14 03:04:37 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:04:37 user nova-compute[71323]:   </features>
Apr 14 03:04:37 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:04:37 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:04:37 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:04:37 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:04:37 user nova-compute[71323]:   </clock>
Apr 14 03:04:37 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:04:37 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:04:37 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:04:37 user nova-compute[71323]:   </cpu>
Apr 14 03:04:37 user nova-compute[71323]:   <devices>
Apr 14 03:04:37 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:04:37 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:37 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:04:37 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:04:37 user nova-compute[71323]:       <serial>423d87a6-a274-408c-ab7c-e91e4edfd963</serial>
Apr 14 03:04:37 user nova-compute[71323]:     </disk>
Apr 14 03:04:37 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:04:37 user nova-compute[71323]:       <mac address="fa:16:3e:20:31:74"/>
Apr 14 03:04:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:04:37 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:04:37 user nova-compute[71323]:       <target dev="tapa07f6861-f9"/>
Apr 14 03:04:37 user nova-compute[71323]:     </interface>
Apr 14 03:04:37 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:04:37 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/62cad020-0344-4e4e-add9-e5bc6175adbb/console.log" append="off"/>
Apr 14 03:04:37 user nova-compute[71323]:     </serial>
Apr 14 03:04:37 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:04:37 user nova-compute[71323]:     <video>
Apr 14 03:04:37 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:04:37 user nova-compute[71323]:     </video>
Apr 14 03:04:37 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:04:37 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:04:37 user nova-compute[71323]:     </rng>
Apr 14 03:04:37 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:04:37 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:04:37 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:04:37 user nova-compute[71323]:     </memballoon>
Apr 14 03:04:37 user nova-compute[71323]:   </devices>
Apr 14 03:04:37 user nova-compute[71323]: </domain>
Apr 14 03:04:37 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-380447828',id=17,image_ref='b58349d2-f4ea-42ed-81b9-5113fd237947',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-nv1ajnpj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-910285030',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:04:25Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=62cad020-0344-4e4e-add9-e5bc6175adbb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG os_vif [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa07f6861-f9, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa07f6861-f9, col_values=(('external_ids', {'iface-id': 'a07f6861-f9f6-42c0-8d36-07db58ad1cb2', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:20:31:74', 'vm-uuid': '62cad020-0344-4e4e-add9-e5bc6175adbb'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:37 user nova-compute[71323]: INFO os_vif [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9')
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:20:31:74, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:04:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441479.4071393, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:04:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Resumed (Lifecycle Event)
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-59db2d22-1c19-49c9-8259-ac89288f2720 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:04:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441479.4090233, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:04:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Started (Lifecycle Event)
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:39 user nova-compute[71323]: WARNING nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:39 user nova-compute[71323]: WARNING nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] No waiting events found dispatching network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:39 user nova-compute[71323]: WARNING nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received unexpected event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 for instance with vm_state building and task_state spawning.
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] No waiting events found dispatching network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:04:39 user nova-compute[71323]: WARNING nova.compute.manager [req-b51adc45-830d-4b02-a6e1-8ba40fa46bed req-dcd234eb-1c6d-4135-85d8-f924c416480f service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received unexpected event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 for instance with vm_state building and task_state spawning.
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441481.0933816, 62cad020-0344-4e4e-add9-e5bc6175adbb => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] VM Resumed (Lifecycle Event)
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Instance spawned successfully.
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441481.0965862, 62cad020-0344-4e4e-add9-e5bc6175adbb => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] VM Started (Lifecycle Event)
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Took 5.16 seconds to spawn the instance on the hypervisor.
Apr 14 03:04:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:04:41 user nova-compute[71323]: INFO nova.compute.manager [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Took 17.02 seconds to build instance.
Apr 14 03:04:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-46cea914-cdf0-4144-b3d4-6fd25dd1a7f7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.117s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:47 user nova-compute[71323]: INFO nova.compute.manager [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Detaching volume f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:47 user nova-compute[71323]: INFO nova.virt.block_device [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attempting to driver detach volume f0c626e1-cd60-46d3-b894-a000a5cb3811 from mountpoint /dev/vdb
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Attempting to detach device vdb from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:47 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:47 user nova-compute[71323]:   <source dev="/dev/sdf"/>
Apr 14 03:04:47 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:47 user nova-compute[71323]:   <serial>f0c626e1-cd60-46d3-b894-a000a5cb3811</serial>
Apr 14 03:04:47 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:04:47 user nova-compute[71323]: </disk>
Apr 14 03:04:47 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully detached device vdb from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the persistent domain config.
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:04:47 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:04:47 user nova-compute[71323]:   <source dev="/dev/sdf"/>
Apr 14 03:04:47 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:04:47 user nova-compute[71323]:   <serial>f0c626e1-cd60-46d3-b894-a000a5cb3811</serial>
Apr 14 03:04:47 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:04:47 user nova-compute[71323]: </disk>
Apr 14 03:04:47 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441487.9294071, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:04:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully detached device vdb from instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 from the live domain config.
Apr 14 03:04:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cpDvVdUM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:04:47 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[92fc8f0a-ea70-4d64-97b3-cf3ed927c882]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:04:47 user nova-compute[71323]: No targets found.
Apr 14 03:04:47 user nova-compute[71323]: iSNS:
Apr 14 03:04:47 user nova-compute[71323]: No targets found.
Apr 14 03:04:47 user nova-compute[71323]: STATIC:
Apr 14 03:04:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:04:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963
Apr 14 03:04:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811
Apr 14 03:04:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:04:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:04:47 user nova-compute[71323]: FIRMWARE:
Apr 14 03:04:47 user nova-compute[71323]: No targets found.
Apr 14 03:04:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:04:47 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:04:47 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:04:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcc42cea-4988-491a-9ab6-299371a5c9c9]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[936b85ee-7308-4636-960b-d29bfc0347fb]: (4, ('tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:48 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:48 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:04:48 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 (non-flash)
Apr 14 03:04:48 user nova-compute[71323]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:04:48 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:04:48 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b007c2320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811'): ({'sdf'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdf {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae2ef5b2-6072-45ad-b194-9a28af095f34]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce377f62-842d-46e6-942e-54dc7398a80e]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdf {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ae47f9a-15f8-4897-8299-933c5206f59b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[10304314-5dd6-491d-a4d9-66da7e1323eb]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[383b2405-2040-4db1-8afa-b5fa3aea08db]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --logout" returned: 0 in 0.051s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bda0b546-7e76-40a6-8ae3-6a8cc9e06e16]: (4, ('Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]\nLogout of [sid: 13, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260]
Apr 14 03:04:48 user nova-compute[71323]: Logout of [sid: 13, target: iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811, portal: 172.16.0.220,3260] successful.
Apr 14 03:04:48 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f0c626e1-cd60-46d3-b894-a000a5cb3811 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[67d1bc99-6ef4-49a8-8fdd-67d3173d870c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.339s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (339ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:04:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:04:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'flavor' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:04:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b267b527-e1c3-47a0-bd22-6a6eed5e6b96 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.208s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:49 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:51 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:04:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:04:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:04:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:04:58 user nova-compute[71323]: INFO nova.compute.claims [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Claim successful on node user
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.758s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:04:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:04:59 user nova-compute[71323]: DEBUG nova.policy [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd666936eb0ee43a2ad9e17a12e6e273d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd2c5b2aafaf4ca7bf62e988ac726658', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:05:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Creating image(s)
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "/opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7771MB free_disk=26.553001403808594GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Successfully created port: 22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f970adb1-fe1b-4b58-83aa-45207a6211ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance dc8c2e40-f896-4d21-9927-67330485d77d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 62cad020-0344-4e4e-add9-e5bc6175adbb actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1920MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk 1073741824" returned: 0 in 0.058s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.211s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Cannot resize image /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'migration_context' on Instance uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.648s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Ensure instance console log exists: /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Successfully updated port: 22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-changed-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Refreshing instance network info cache due to event network-changed-22f727c9-9207-46e6-a471-48de5bf03677. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Instance network_info: |[{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.neutron [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Refreshing network info cache for port 22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Start _get_guest_xml network_info=[{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:05:01 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:01 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-514488998',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-514488998',id=18,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-8i0b0x43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:05:00Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:05:01 user nova-compute[71323]:   <uuid>d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e</uuid>
Apr 14 03:05:01 user nova-compute[71323]:   <name>instance-00000012</name>
Apr 14 03:05:01 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:05:01 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:05:01 user nova-compute[71323]:   <metadata>
Apr 14 03:05:01 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:05:01 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-514488998</nova:name>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:05:01</nova:creationTime>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:05:01 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:05:01 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:owner>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:05:01 user nova-compute[71323]:       </nova:owner>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:05:01 user nova-compute[71323]:       <nova:ports>
Apr 14 03:05:01 user nova-compute[71323]:         <nova:port uuid="22f727c9-9207-46e6-a471-48de5bf03677">
Apr 14 03:05:01 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 03:05:01 user nova-compute[71323]:         </nova:port>
Apr 14 03:05:01 user nova-compute[71323]:       </nova:ports>
Apr 14 03:05:01 user nova-compute[71323]:     </nova:instance>
Apr 14 03:05:01 user nova-compute[71323]:   </metadata>
Apr 14 03:05:01 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:05:01 user nova-compute[71323]:     <system>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="serial">d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e</entry>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="uuid">d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e</entry>
Apr 14 03:05:01 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:05:01 user nova-compute[71323]:     </system>
Apr 14 03:05:01 user nova-compute[71323]:   </sysinfo>
Apr 14 03:05:01 user nova-compute[71323]:   <os>
Apr 14 03:05:01 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:05:01 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:05:01 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:05:01 user nova-compute[71323]:   </os>
Apr 14 03:05:01 user nova-compute[71323]:   <features>
Apr 14 03:05:01 user nova-compute[71323]:     <acpi/>
Apr 14 03:05:01 user nova-compute[71323]:     <apic/>
Apr 14 03:05:01 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:05:01 user nova-compute[71323]:   </features>
Apr 14 03:05:01 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:05:01 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:05:01 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:05:01 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:05:01 user nova-compute[71323]:   </clock>
Apr 14 03:05:01 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:05:01 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:05:01 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:05:01 user nova-compute[71323]:   </cpu>
Apr 14 03:05:01 user nova-compute[71323]:   <devices>
Apr 14 03:05:01 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:05:01 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:05:01 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk"/>
Apr 14 03:05:01 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:05:01 user nova-compute[71323]:     </disk>
Apr 14 03:05:01 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:05:01 user nova-compute[71323]:       <mac address="fa:16:3e:db:67:a6"/>
Apr 14 03:05:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:01 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:05:01 user nova-compute[71323]:       <target dev="tap22f727c9-92"/>
Apr 14 03:05:01 user nova-compute[71323]:     </interface>
Apr 14 03:05:01 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:05:01 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/console.log" append="off"/>
Apr 14 03:05:01 user nova-compute[71323]:     </serial>
Apr 14 03:05:01 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:05:01 user nova-compute[71323]:     <video>
Apr 14 03:05:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:01 user nova-compute[71323]:     </video>
Apr 14 03:05:01 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:05:01 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:05:01 user nova-compute[71323]:     </rng>
Apr 14 03:05:01 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:05:01 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:05:01 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:05:01 user nova-compute[71323]:     </memballoon>
Apr 14 03:05:01 user nova-compute[71323]:   </devices>
Apr 14 03:05:01 user nova-compute[71323]: </domain>
Apr 14 03:05:01 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-514488998',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-514488998',id=18,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-8i0b0x43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:05:00Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG os_vif [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap22f727c9-92, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap22f727c9-92, col_values=(('external_ids', {'iface-id': '22f727c9-9207-46e6-a471-48de5bf03677', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:db:67:a6', 'vm-uuid': 'd7277eb6-c2d1-4c3a-8986-51d3fbf1c29e'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:01 user nova-compute[71323]: INFO os_vif [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92')
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:05:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:db:67:a6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:05:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updated VIF entry in instance network info cache for port 22f727c9-9207-46e6-a471-48de5bf03677. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:05:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13fa6735-2db9-4dac-872a-b1bf3746b060 req-4ed684c9-0759-4586-a9e2-40cb7f62d153 service nova] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] No waiting events found dispatching network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:03 user nova-compute[71323]: WARNING nova.compute.manager [req-522ea051-a34f-444a-acba-fb3d21edfe59 req-6fcdac54-39de-499a-84e4-e1ed45043c6a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received unexpected event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 for instance with vm_state building and task_state spawning.
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] No waiting events found dispatching network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:05 user nova-compute[71323]: WARNING nova.compute.manager [req-a1bd443e-625e-44cd-88d4-c6b024adc147 req-d5b5f137-3570-4194-93a5-c84d878ae67d service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received unexpected event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 for instance with vm_state building and task_state spawning.
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441505.4294262, d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] VM Resumed (Lifecycle Event)
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Instance spawned successfully.
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441505.4323363, d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] VM Started (Lifecycle Event)
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Took 5.42 seconds to spawn the instance on the hypervisor.
Apr 14 03:05:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:05:05 user nova-compute[71323]: INFO nova.compute.manager [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Took 6.99 seconds to build instance.
Apr 14 03:05:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-bcda52d9-9dc7-4c79-95f9-2689b5d2abd8 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.095s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:10 user nova-compute[71323]: INFO nova.compute.manager [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Terminating instance
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-unplugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] No waiting events found dispatching network-vif-unplugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-1c860dd8-0abd-4985-b8fa-f76633b5c99a req-0c7a39e5-9a95-4f44-9423-0fc42b1a53a1 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-unplugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:05:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Instance destroyed successfully.
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.objects.instance [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lazy-loading 'resources' on Instance uuid b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:01:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1072264904',display_name='tempest-TestVolumeSwap-server-1072264904',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1072264904',id=14,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBnYqF9Yc0//Xc2kqydIgMZFcAkhLZn1nUjuDCDft78aomgzO7Y5n1Hl9qALLn2itHKhHsgBqDZqHr/4EY/d5MXpZoR4+r1CWeR2M7mZVFWbBOGHV8B0e9m5AsRBcS+9/g==',key_name='tempest-keypair-603146841',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f987064bf0ed4a2fa8714064fe8b43c8',ramdisk_id='',reservation_id='r-qoounn30',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeSwap-1328008515',owner_user_name='tempest-TestVolumeSwap-1328008515-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f089c5c093754f8082181a8fc1c002f4',uuid=b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.49", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converting VIF {"id": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "address": "fa:16:3e:2b:e1:ca", "network": {"id": "c5b60b86-e5de-4981-96d7-316ce84984f2", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1984856349-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.49", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f987064bf0ed4a2fa8714064fe8b43c8", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb975970e-4c", "ovs_interfaceid": "b975970e-4ce0-4b1c-b1b3-956befa892e0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG os_vif [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb975970e-4c, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:10 user nova-compute[71323]: INFO os_vif [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2b:e1:ca,bridge_name='br-int',has_traffic_filtering=True,id=b975970e-4ce0-4b1c-b1b3-956befa892e0,network=Network(c5b60b86-e5de-4981-96d7-316ce84984f2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb975970e-4c')
Apr 14 03:05:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Deleting instance files /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49_del
Apr 14 03:05:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Deletion of /opt/stack/data/nova/instances/b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49_del complete
Apr 14 03:05:10 user nova-compute[71323]: INFO nova.compute.manager [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 14 03:05:10 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:05:10 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:05:11 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:11 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Took 0.96 seconds to deallocate network for instance.
Apr 14 03:05:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-31f8136f-df60-442a-afe2-c39280f71616 req-9f770dcf-9430-44c3-987c-96bea3ff2dfa service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-deleted-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.409s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:12 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Deleted allocations for instance b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49
Apr 14 03:05:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-43397368-d806-4962-88a9-24f9e8d2b934 tempest-TestVolumeSwap-1328008515 tempest-TestVolumeSwap-1328008515-project-member] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.428s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] Acquiring lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] Lock "b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] No waiting events found dispatching network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:12 user nova-compute[71323]: WARNING nova.compute.manager [req-b88572a8-116a-4acd-bb08-03476f86c70d req-b1f910f7-cb03-487d-a635-f1d766cc54c9 service nova] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Received unexpected event network-vif-plugged-b975970e-4ce0-4b1c-b1b3-956befa892e0 for instance with vm_state deleted and task_state None.
Apr 14 03:05:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:05:24 user nova-compute[71323]: INFO nova.compute.claims [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Claim successful on node user
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.388s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:05:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:05:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Creating image(s)
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG nova.policy [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17999fdd4eb449d49d74b4de20f6614a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a991a4fd13b4a819b24479a341ba08c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk 1073741824" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.198s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing instance network info cache due to event network-changed-a36f717a-9a8b-43c9-ba01-0f03a522d166. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] Acquiring lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] Acquired lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Refreshing network info cache for port a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:25 user nova-compute[71323]: INFO nova.compute.manager [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Terminating instance
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Cannot resize image /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'migration_context' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Ensure instance console log exists: /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Successfully created port: fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-unplugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] No waiting events found dispatching network-vif-unplugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-76fb2917-a544-45fb-be06-3b07a5a7cefc req-e718baca-8592-4637-9b05-80d6bda36743 service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-unplugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updated VIF entry in instance network info cache for port a36f717a-9a8b-43c9-ba01-0f03a522d166. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating instance_info_cache with network_info: [{"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ca32217e-b3a3-4ae1-b7f2-0043d20fbea0 req-b847c2b6-a9cc-48a1-9dc9-6bba92ef5adf service nova] Releasing lock "refresh_cache-dc8c2e40-f896-4d21-9927-67330485d77d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441510.787267, b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:25 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] VM Stopped (Lifecycle Event)
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ee931cd9-e729-4804-913f-4a76c956f2c1 None None] [instance: b06dcbb8-bb2d-4789-9e0a-0388cf4c9b49] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Instance destroyed successfully.
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'resources' on Instance uuid dc8c2e40-f896-4d21-9927-67330485d77d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-653907637',display_name='tempest-TestStampPattern-server-653907637',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-653907637',id=16,image_ref='a7190884-2b37-45b0-a8e1-4e55a042e5ea',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cc0ip46y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='2ca84990-b165-4d0b-a917-307cd8f96f28',image_min_disk='1',image_min_ram='0',image_owner_id='6371b4c05df843da9b134e12994c1d7a',image_owner_project_name='tempest-TestStampPattern-323884970',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-323884970-project-member',image_user_id='337e2d8118324215b5a0baa36a5895b0',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:42Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=dc8c2e40-f896-4d21-9927-67330485d77d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.7", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "address": "fa:16:3e:ab:25:10", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.7", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa36f717a-9a", "ovs_interfaceid": "a36f717a-9a8b-43c9-ba01-0f03a522d166", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG os_vif [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa36f717a-9a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:26 user nova-compute[71323]: INFO os_vif [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ab:25:10,bridge_name='br-int',has_traffic_filtering=True,id=a36f717a-9a8b-43c9-ba01-0f03a522d166,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa36f717a-9a')
Apr 14 03:05:26 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Deleting instance files /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d_del
Apr 14 03:05:26 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Deletion of /opt/stack/data/nova/instances/dc8c2e40-f896-4d21-9927-67330485d77d_del complete
Apr 14 03:05:26 user nova-compute[71323]: INFO nova.compute.manager [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Took 0.86 seconds to destroy the instance on the hypervisor.
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Successfully updated port: fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:26 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Took 0.51 seconds to deallocate network for instance.
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Instance network_info: |[{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Start _get_guest_xml network_info=[{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:05:26 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:26 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:05:25Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'pci_devices' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:05:26 user nova-compute[71323]:   <uuid>3e73add1-26be-47dd-adc5-506ab517544a</uuid>
Apr 14 03:05:26 user nova-compute[71323]:   <name>instance-00000013</name>
Apr 14 03:05:26 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:05:26 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:05:26 user nova-compute[71323]:   <metadata>
Apr 14 03:05:26 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:05:26 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:05:26</nova:creationTime>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:05:26 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:05:26 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:owner>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:05:26 user nova-compute[71323]:       </nova:owner>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:05:26 user nova-compute[71323]:       <nova:ports>
Apr 14 03:05:26 user nova-compute[71323]:         <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:05:26 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:05:26 user nova-compute[71323]:         </nova:port>
Apr 14 03:05:26 user nova-compute[71323]:       </nova:ports>
Apr 14 03:05:26 user nova-compute[71323]:     </nova:instance>
Apr 14 03:05:26 user nova-compute[71323]:   </metadata>
Apr 14 03:05:26 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:05:26 user nova-compute[71323]:     <system>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="serial">3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="uuid">3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:05:26 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:05:26 user nova-compute[71323]:     </system>
Apr 14 03:05:26 user nova-compute[71323]:   </sysinfo>
Apr 14 03:05:26 user nova-compute[71323]:   <os>
Apr 14 03:05:26 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:05:26 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:05:26 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:05:26 user nova-compute[71323]:   </os>
Apr 14 03:05:26 user nova-compute[71323]:   <features>
Apr 14 03:05:26 user nova-compute[71323]:     <acpi/>
Apr 14 03:05:26 user nova-compute[71323]:     <apic/>
Apr 14 03:05:26 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:05:26 user nova-compute[71323]:   </features>
Apr 14 03:05:26 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:05:26 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:05:26 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:05:26 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:05:26 user nova-compute[71323]:   </clock>
Apr 14 03:05:26 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:05:26 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:05:26 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:05:26 user nova-compute[71323]:   </cpu>
Apr 14 03:05:26 user nova-compute[71323]:   <devices>
Apr 14 03:05:26 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:05:26 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:05:26 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk"/>
Apr 14 03:05:26 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:05:26 user nova-compute[71323]:     </disk>
Apr 14 03:05:26 user nova-compute[71323]:     <disk type="file" device="cdrom">
Apr 14 03:05:26 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 03:05:26 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config"/>
Apr 14 03:05:26 user nova-compute[71323]:       <target dev="hda" bus="ide"/>
Apr 14 03:05:26 user nova-compute[71323]:     </disk>
Apr 14 03:05:26 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:05:26 user nova-compute[71323]:       <mac address="fa:16:3e:e7:c3:de"/>
Apr 14 03:05:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:26 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:05:26 user nova-compute[71323]:       <target dev="tapfb9a2be7-8d"/>
Apr 14 03:05:26 user nova-compute[71323]:     </interface>
Apr 14 03:05:26 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:05:26 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log" append="off"/>
Apr 14 03:05:26 user nova-compute[71323]:     </serial>
Apr 14 03:05:26 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:05:26 user nova-compute[71323]:     <video>
Apr 14 03:05:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:26 user nova-compute[71323]:     </video>
Apr 14 03:05:26 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:05:26 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:05:26 user nova-compute[71323]:     </rng>
Apr 14 03:05:26 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:05:26 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:05:26 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:05:26 user nova-compute[71323]:     </memballoon>
Apr 14 03:05:26 user nova-compute[71323]:   </devices>
Apr 14 03:05:26 user nova-compute[71323]: </domain>
Apr 14 03:05:26 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:05:25Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG os_vif [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfb9a2be7-8d, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapfb9a2be7-8d, col_values=(('external_ids', {'iface-id': 'fb9a2be7-8d62-4528-a23c-bfdf995bd6dc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e7:c3:de', 'vm-uuid': '3e73add1-26be-47dd-adc5-506ab517544a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:26 user nova-compute[71323]: INFO os_vif [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d')
Apr 14 03:05:26 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name hda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No VIF found with MAC fa:16:3e:e7:c3:de, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:05:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Using config drive
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.407s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:27 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Deleted allocations for instance dc8c2e40-f896-4d21-9927-67330485d77d
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-changed-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing instance network info cache due to event network-changed-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing network info cache for port fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7118858f-2be2-4280-8a4f-d62ad66b5846 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "dc8c2e40-f896-4d21-9927-67330485d77d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.978s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Creating config drive at /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpda3tcz2k {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpda3tcz2k" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updated VIF entry in instance network info cache for port fb9a2be7-8d62-4528-a23c-bfdf995bd6dc. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e9f3f15f-d065-4b07-8d38-4b10ec805ba4 req-552d9e37-4fa4-4940-8886-95db88822e22 service nova] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] Acquiring lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] Lock "dc8c2e40-f896-4d21-9927-67330485d77d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] No waiting events found dispatching network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:27 user nova-compute[71323]: WARNING nova.compute.manager [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received unexpected event network-vif-plugged-a36f717a-9a8b-43c9-ba01-0f03a522d166 for instance with vm_state deleted and task_state None.
Apr 14 03:05:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8d8a6f7-50ab-4eb5-ae2d-9bd31e07b5a3 req-3308cb81-4714-4c68-8fc9-65dd7c323eec service nova] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Received event network-vif-deleted-a36f717a-9a8b-43c9-ba01-0f03a522d166 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:29 user nova-compute[71323]: WARNING nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc for instance with vm_state building and task_state spawning.
Apr 14 03:05:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:29 user nova-compute[71323]: WARNING nova.compute.manager [req-f2bf238d-675e-4aea-ba2e-31ea522e275e req-f3b7ac93-9f4a-4354-8dc3-397c68bc4ec3 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc for instance with vm_state building and task_state spawning.
Apr 14 03:05:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441530.6584733, 3e73add1-26be-47dd-adc5-506ab517544a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] VM Resumed (Lifecycle Event)
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Instance spawned successfully.
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441530.661995, 3e73add1-26be-47dd-adc5-506ab517544a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] VM Started (Lifecycle Event)
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Took 6.13 seconds to spawn the instance on the hypervisor.
Apr 14 03:05:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:30 user nova-compute[71323]: INFO nova.compute.manager [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Took 6.87 seconds to build instance.
Apr 14 03:05:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dd6b45f-789e-4cb1-ba44-7e1503fcf3e5 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.964s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:05:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'info_cache' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance destroyed successfully.
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'resources' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:03:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG os_vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb85a839f-97, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:33 user nova-compute[71323]: INFO os_vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start _get_guest_xml network_info=[{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'numa_topology' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:05:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'vcpu_model' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:03:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'pci_devices' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:05:34 user nova-compute[71323]:   <uuid>bbb6cd58-ce7a-43ae-876d-51a888193f60</uuid>
Apr 14 03:05:34 user nova-compute[71323]:   <name>instance-00000002</name>
Apr 14 03:05:34 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:05:34 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:05:34 user nova-compute[71323]:   <metadata>
Apr 14 03:05:34 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:05:34 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeTestJSON-server-1854200462</nova:name>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:05:33</nova:creationTime>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:05:34 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:05:34 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:owner>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:user uuid="d6d78d4991b5493e8893034e91ed407b">tempest-AttachVolumeTestJSON-975876676-project-member</nova:user>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:project uuid="130665008cf648cf87467ae2777a9170">tempest-AttachVolumeTestJSON-975876676</nova:project>
Apr 14 03:05:34 user nova-compute[71323]:       </nova:owner>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:05:34 user nova-compute[71323]:       <nova:ports>
Apr 14 03:05:34 user nova-compute[71323]:         <nova:port uuid="b85a839f-973d-415e-94b6-cdb480a8b89d">
Apr 14 03:05:34 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:05:34 user nova-compute[71323]:         </nova:port>
Apr 14 03:05:34 user nova-compute[71323]:       </nova:ports>
Apr 14 03:05:34 user nova-compute[71323]:     </nova:instance>
Apr 14 03:05:34 user nova-compute[71323]:   </metadata>
Apr 14 03:05:34 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:05:34 user nova-compute[71323]:     <system>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="serial">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="uuid">bbb6cd58-ce7a-43ae-876d-51a888193f60</entry>
Apr 14 03:05:34 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:05:34 user nova-compute[71323]:     </system>
Apr 14 03:05:34 user nova-compute[71323]:   </sysinfo>
Apr 14 03:05:34 user nova-compute[71323]:   <os>
Apr 14 03:05:34 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:05:34 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:05:34 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:05:34 user nova-compute[71323]:   </os>
Apr 14 03:05:34 user nova-compute[71323]:   <features>
Apr 14 03:05:34 user nova-compute[71323]:     <acpi/>
Apr 14 03:05:34 user nova-compute[71323]:     <apic/>
Apr 14 03:05:34 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:05:34 user nova-compute[71323]:   </features>
Apr 14 03:05:34 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:05:34 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:05:34 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:05:34 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:05:34 user nova-compute[71323]:   </clock>
Apr 14 03:05:34 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:05:34 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:05:34 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:05:34 user nova-compute[71323]:   </cpu>
Apr 14 03:05:34 user nova-compute[71323]:   <devices>
Apr 14 03:05:34 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:05:34 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:05:34 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk"/>
Apr 14 03:05:34 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:05:34 user nova-compute[71323]:     </disk>
Apr 14 03:05:34 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:05:34 user nova-compute[71323]:       <mac address="fa:16:3e:f7:89:a6"/>
Apr 14 03:05:34 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:34 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:05:34 user nova-compute[71323]:       <target dev="tapb85a839f-97"/>
Apr 14 03:05:34 user nova-compute[71323]:     </interface>
Apr 14 03:05:34 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:05:34 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/console.log" append="off"/>
Apr 14 03:05:34 user nova-compute[71323]:     </serial>
Apr 14 03:05:34 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:05:34 user nova-compute[71323]:     <video>
Apr 14 03:05:34 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:05:34 user nova-compute[71323]:     </video>
Apr 14 03:05:34 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:05:34 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:05:34 user nova-compute[71323]:     </rng>
Apr 14 03:05:34 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:05:34 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:05:34 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:05:34 user nova-compute[71323]:     </memballoon>
Apr 14 03:05:34 user nova-compute[71323]:   </devices>
Apr 14 03:05:34 user nova-compute[71323]: </domain>
Apr 14 03:05:34 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.160s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'trusted_certs' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking if we can resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Cannot resize image /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'migration_context' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:03:43Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG os_vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb85a839f-97, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb85a839f-97, col_values=(('external_ids', {'iface-id': 'b85a839f-973d-415e-94b6-cdb480a8b89d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f7:89:a6', 'vm-uuid': 'bbb6cd58-ce7a-43ae-876d-51a888193f60'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:05:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:34 user nova-compute[71323]: INFO os_vif [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:36 user nova-compute[71323]: WARNING nova.compute.manager [req-a419e870-f9d5-4cbb-a16f-8f563fcac8eb req-7f178801-1ae8-483d-9d30-b1768db4ec61 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state stopped and task_state powering-on.
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441538.4231372, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Resumed (Lifecycle Event)
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:05:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance rebooted successfully.
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5dfe1a6b-7e20-4230-aa80-25dab14b96f1 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] During sync_power_state the instance has a pending task (powering-on). Skip.
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441538.4272149, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Started (Lifecycle Event)
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Synchronizing instance power state after lifecycle event "Started"; current vm_state: stopped, current task_state: powering-on, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:05:38 user nova-compute[71323]: WARNING nova.compute.manager [req-0b455d2c-e384-4418-ad2d-436f56f1f5ff req-4cc087f1-0afb-4d40-822b-5cf61b0f9d79 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state None.
Apr 14 03:05:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:40 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441525.9584856, dc8c2e40-f896-4d21-9927-67330485d77d => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:05:40 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] VM Stopped (Lifecycle Event)
Apr 14 03:05:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-60bfe72d-879c-46f0-8c0d-1be0dc3042c9 None None] [instance: dc8c2e40-f896-4d21-9927-67330485d77d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:05:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:05:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:05:56 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:05:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updating instance_info_cache with network_info: [{"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:05:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-1fb18db3-28b0-4886-a680-4c534d31cb6b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:05:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:05:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:05:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:05:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:06:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7647MB free_disk=26.527923583984375GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1fb18db3-28b0-4886-a680-4c534d31cb6b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2ca84990-b165-4d0b-a917-307cd8f96f28 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f970adb1-fe1b-4b58-83aa-45207a6211ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 62cad020-0344-4e4e-add9-e5bc6175adbb actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3e73add1-26be-47dd-adc5-506ab517544a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:06:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.460s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:14 user nova-compute[71323]: INFO nova.compute.manager [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Terminating instance
Apr 14 03:06:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-unplugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] No waiting events found dispatching network-vif-unplugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-ec2e6e73-31e4-4a58-8c1c-6fe42a39562b req-313a5ba2-19cd-46a6-9e71-f1f909f1c05a service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-unplugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Instance destroyed successfully.
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.objects.instance [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid 62cad020-0344-4e4e-add9-e5bc6175adbb {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-380447828',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-380447828',id=17,image_ref='b58349d2-f4ea-42ed-81b9-5113fd237947',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:04:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-nv1ajnpj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-910285030',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member',image_root_device_name='/dev/vda',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:04:41Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=62cad020-0344-4e4e-add9-e5bc6175adbb,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "address": "fa:16:3e:20:31:74", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa07f6861-f9", "ovs_interfaceid": "a07f6861-f9f6-42c0-8d36-07db58ad1cb2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_vif [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa07f6861-f9, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:15 user nova-compute[71323]: INFO os_vif [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:20:31:74,bridge_name='br-int',has_traffic_filtering=True,id=a07f6861-f9f6-42c0-8d36-07db58ad1cb2,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa07f6861-f9')
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ImLoQRXC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:15 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ba9a893-e980-46ee-a60e-14963d2ce7bc]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:06:15 user nova-compute[71323]: No targets found.
Apr 14 03:06:15 user nova-compute[71323]: iSNS:
Apr 14 03:06:15 user nova-compute[71323]: No targets found.
Apr 14 03:06:15 user nova-compute[71323]: STATIC:
Apr 14 03:06:15 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:06:15 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:06:15 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:06:15 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963
Apr 14 03:06:15 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:06:15 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:06:15 user nova-compute[71323]: FIRMWARE:
Apr 14 03:06:15 user nova-compute[71323]: No targets found.
Apr 14 03:06:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:06:15 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:06:15 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de89ae04-80b7-498e-be2f-d42abe619d49]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[79f5e615-8d71-453d-bb66-b18b4ceaa3c7]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:15 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:06:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:15 user nova-compute[71323]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 (non-flash)
Apr 14 03:06:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae01bbac0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c9c01a4-295e-4696-a429-b1cb46e525b6]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1af34352-b24d-4be3-a12e-8c68b546ac9a]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[023e7d7a-f87a-4e60-8450-46f4d87dbffc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.047s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f32e7e9-baab-4dc0-b6a3-67469f3a0ccb]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[86c7b057-039b-4631-bda4-9ddbc555a5d1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[beba0ca6-908f-4983-804b-5cb28ba71e06]: (4, ('Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260]\nLogout of [sid: 19, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260]
Apr 14 03:06:15 user nova-compute[71323]: Logout of [sid: 19, target: iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963, portal: 172.16.0.220,3260] successful.
Apr 14 03:06:15 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:423d87a6-a274-408c-ab7c-e91e4edfd963 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[52c155dc-f995-4e43-90cd-260365701d07]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.317s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (318ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:06:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Deleting instance files /opt/stack/data/nova/instances/62cad020-0344-4e4e-add9-e5bc6175adbb_del
Apr 14 03:06:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Deletion of /opt/stack/data/nova/instances/62cad020-0344-4e4e-add9-e5bc6175adbb_del complete
Apr 14 03:06:15 user nova-compute[71323]: INFO nova.compute.manager [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Took 1.18 seconds to destroy the instance on the hypervisor.
Apr 14 03:06:15 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:06:15 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:06:16 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:16 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Took 0.63 seconds to deallocate network for instance.
Apr 14 03:06:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0c53d5f-6e29-41ba-9172-0111706416ef req-c53cf943-f599-4c3f-9132-18834118c4c2 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-deleted-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] Acquiring lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] No waiting events found dispatching network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:17 user nova-compute[71323]: WARNING nova.compute.manager [req-efd4f746-0c93-45f9-8969-4c3f344336f9 req-0799ae4b-b651-450f-bfb8-0cedc9b41c86 service nova] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Received unexpected event network-vif-plugged-a07f6861-f9f6-42c0-8d36-07db58ad1cb2 for instance with vm_state active and task_state deleting.
Apr 14 03:06:17 user nova-compute[71323]: INFO nova.compute.manager [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Took 0.80 seconds to detach 1 volumes for instance.
Apr 14 03:06:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Deleting volume: 423d87a6-a274-408c-ab7c-e91e4edfd963 {{(pid=71323) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.326s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:17 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance 62cad020-0344-4e4e-add9-e5bc6175adbb
Apr 14 03:06:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-502bcf7c-014b-4c72-968d-d7b5bb93e5c7 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "62cad020-0344-4e4e-add9-e5bc6175adbb" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.219s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing instance network info cache due to event network-changed-920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] Acquiring lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] Acquired lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Refreshing network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:24 user nova-compute[71323]: INFO nova.compute.manager [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Terminating instance
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updated VIF entry in instance network info cache for port 920aad6f-1d46-48d5-81eb-95ffbc851da0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating instance_info_cache with network_info: [{"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Instance destroyed successfully.
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.objects.instance [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lazy-loading 'resources' on Instance uuid 2ca84990-b165-4d0b-a917-307cd8f96f28 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:59:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-491407941',display_name='tempest-TestStampPattern-server-491407941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-teststamppattern-server-491407941',id=9,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBOvsWjeP7/S815tfdb/cOxId+DiLVdU5d0FX5X3LVhv2hHJTEUDK11dS58p8M2b3SiTe14kIsNXEkLkLz6Nb9uJ4UZw0ejuGhB6i4/LUSktkX9meq382hMjQRuSfxbyUw==',key_name='tempest-TestStampPattern-1951210375',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:59:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6371b4c05df843da9b134e12994c1d7a',ramdisk_id='',reservation_id='r-cv7ox6d1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestStampPattern-323884970',owner_user_name='tempest-TestStampPattern-323884970-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:27Z,user_data=None,user_id='337e2d8118324215b5a0baa36a5895b0',uuid=2ca84990-b165-4d0b-a917-307cd8f96f28,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.246", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converting VIF {"id": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "address": "fa:16:3e:93:69:3c", "network": {"id": "516bab80-c2e3-4a1c-9181-5eae563a4ae5", "bridge": "br-int", "label": "tempest-TestStampPattern-505191750-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.246", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6371b4c05df843da9b134e12994c1d7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap920aad6f-1d", "ovs_interfaceid": "920aad6f-1d46-48d5-81eb-95ffbc851da0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG os_vif [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap920aad6f-1d, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4dca01c2-05d7-4eac-b23d-56e3c023d056 req-37988397-38d4-4837-8998-1d9e4caada41 service nova] Releasing lock "refresh_cache-2ca84990-b165-4d0b-a917-307cd8f96f28" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:06:25 user nova-compute[71323]: INFO os_vif [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:93:69:3c,bridge_name='br-int',has_traffic_filtering=True,id=920aad6f-1d46-48d5-81eb-95ffbc851da0,network=Network(516bab80-c2e3-4a1c-9181-5eae563a4ae5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap920aad6f-1d')
Apr 14 03:06:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Deleting instance files /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28_del
Apr 14 03:06:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Deletion of /opt/stack/data/nova/instances/2ca84990-b165-4d0b-a917-307cd8f96f28_del complete
Apr 14 03:06:25 user nova-compute[71323]: INFO nova.compute.manager [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Took 0.89 seconds to destroy the instance on the hypervisor.
Apr 14 03:06:25 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:06:25 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:06:25 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:06:25 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:25 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Took 0.59 seconds to deallocate network for instance.
Apr 14 03:06:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:25 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.325s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:26 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Deleted allocations for instance 2ca84990-b165-4d0b-a917-307cd8f96f28
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-vif-unplugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] No waiting events found dispatching network-vif-unplugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received unexpected event network-vif-unplugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 for instance with vm_state deleted and task_state None.
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Acquiring lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] No waiting events found dispatching network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received unexpected event network-vif-plugged-920aad6f-1d46-48d5-81eb-95ffbc851da0 for instance with vm_state deleted and task_state None.
Apr 14 03:06:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e4c5e27e-06fe-42a7-be0f-f3b27d148907 req-8af9cada-e8a3-4edb-ab90-4129a8347658 service nova] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Received event network-vif-deleted-920aad6f-1d46-48d5-81eb-95ffbc851da0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1d042601-8d0f-4eff-82f1-0a8a07e7b115 tempest-TestStampPattern-323884970 tempest-TestStampPattern-323884970-project-member] Lock "2ca84990-b165-4d0b-a917-307cd8f96f28" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.010s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441575.3767343, 62cad020-0344-4e4e-add9-e5bc6175adbb => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:06:30 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] VM Stopped (Lifecycle Event)
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-913d858c-b820-4e66-86ba-97ab617ae32d None None] [instance: 62cad020-0344-4e4e-add9-e5bc6175adbb] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:06:30 user nova-compute[71323]: INFO nova.compute.claims [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Claim successful on node user
Apr 14 03:06:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:30 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.421s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:06:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.policy [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd5e8150f1502427fb3148f20354d212e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '558bfd7e1d9049d396381de148b0c4a5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:06:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Creating image(s)
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.168s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk 1073741824" returned: 0 in 0.062s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.239s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Checking if we can resize image /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Cannot resize image /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Successfully created port: 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Ensure instance console log exists: /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Successfully updated port: 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing instance network info cache due to event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance network_info: |[{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start _get_guest_xml network_info=[{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:06:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:06:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:06:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-413600185',display_name='tempest-ServerRescueNegativeTestJSON-server-413600185',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-413600185',id=20,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNRt1vkGxga6d2lnbq6r7KuWDr3P9XxrdKyfybFxOJEVdZ9m8h/yMxKRYhp7IzJglBQvcA3Veeq6pJJUHOvucLytIWZV1lh6jcIrnJ6ba4rOplP1XVeraWzy0zv10BSYng==',key_name='tempest-keypair-158606901',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-jnxbele2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d5e8150f1502427fb3148f20354d212e',uuid=4e47adee-bd94-4655-9bbd-ed59c5478531,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:06:33 user nova-compute[71323]:   <uuid>4e47adee-bd94-4655-9bbd-ed59c5478531</uuid>
Apr 14 03:06:33 user nova-compute[71323]:   <name>instance-00000014</name>
Apr 14 03:06:33 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:06:33 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:06:33 user nova-compute[71323]:   <metadata>
Apr 14 03:06:33 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:06:33 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-413600185</nova:name>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:06:33</nova:creationTime>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:06:33 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:06:33 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:owner>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 03:06:33 user nova-compute[71323]:       </nova:owner>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:06:33 user nova-compute[71323]:       <nova:ports>
Apr 14 03:06:33 user nova-compute[71323]:         <nova:port uuid="4773c5d2-1520-4fc1-ae46-f83a883cfd1b">
Apr 14 03:06:33 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:06:33 user nova-compute[71323]:         </nova:port>
Apr 14 03:06:33 user nova-compute[71323]:       </nova:ports>
Apr 14 03:06:33 user nova-compute[71323]:     </nova:instance>
Apr 14 03:06:33 user nova-compute[71323]:   </metadata>
Apr 14 03:06:33 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:06:33 user nova-compute[71323]:     <system>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="serial">4e47adee-bd94-4655-9bbd-ed59c5478531</entry>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="uuid">4e47adee-bd94-4655-9bbd-ed59c5478531</entry>
Apr 14 03:06:33 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:06:33 user nova-compute[71323]:     </system>
Apr 14 03:06:33 user nova-compute[71323]:   </sysinfo>
Apr 14 03:06:33 user nova-compute[71323]:   <os>
Apr 14 03:06:33 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:06:33 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:06:33 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:06:33 user nova-compute[71323]:   </os>
Apr 14 03:06:33 user nova-compute[71323]:   <features>
Apr 14 03:06:33 user nova-compute[71323]:     <acpi/>
Apr 14 03:06:33 user nova-compute[71323]:     <apic/>
Apr 14 03:06:33 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:06:33 user nova-compute[71323]:   </features>
Apr 14 03:06:33 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:06:33 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:06:33 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:06:33 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:06:33 user nova-compute[71323]:   </clock>
Apr 14 03:06:33 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:06:33 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:06:33 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:06:33 user nova-compute[71323]:   </cpu>
Apr 14 03:06:33 user nova-compute[71323]:   <devices>
Apr 14 03:06:33 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:06:33 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:06:33 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk"/>
Apr 14 03:06:33 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:06:33 user nova-compute[71323]:     </disk>
Apr 14 03:06:33 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:06:33 user nova-compute[71323]:       <mac address="fa:16:3e:3f:e5:e9"/>
Apr 14 03:06:33 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:06:33 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:06:33 user nova-compute[71323]:       <target dev="tap4773c5d2-15"/>
Apr 14 03:06:33 user nova-compute[71323]:     </interface>
Apr 14 03:06:33 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:06:33 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/console.log" append="off"/>
Apr 14 03:06:33 user nova-compute[71323]:     </serial>
Apr 14 03:06:33 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:06:33 user nova-compute[71323]:     <video>
Apr 14 03:06:33 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:06:33 user nova-compute[71323]:     </video>
Apr 14 03:06:33 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:06:33 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:06:33 user nova-compute[71323]:     </rng>
Apr 14 03:06:33 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:06:33 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:06:33 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:06:33 user nova-compute[71323]:     </memballoon>
Apr 14 03:06:33 user nova-compute[71323]:   </devices>
Apr 14 03:06:33 user nova-compute[71323]: </domain>
Apr 14 03:06:33 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:06:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-413600185',display_name='tempest-ServerRescueNegativeTestJSON-server-413600185',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-413600185',id=20,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNRt1vkGxga6d2lnbq6r7KuWDr3P9XxrdKyfybFxOJEVdZ9m8h/yMxKRYhp7IzJglBQvcA3Veeq6pJJUHOvucLytIWZV1lh6jcIrnJ6ba4rOplP1XVeraWzy0zv10BSYng==',key_name='tempest-keypair-158606901',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-jnxbele2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d5e8150f1502427fb3148f20354d212e',uuid=4e47adee-bd94-4655-9bbd-ed59c5478531,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG os_vif [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4773c5d2-15, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4773c5d2-15, col_values=(('external_ids', {'iface-id': '4773c5d2-1520-4fc1-ae46-f83a883cfd1b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3f:e5:e9', 'vm-uuid': '4e47adee-bd94-4655-9bbd-ed59c5478531'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:33 user nova-compute[71323]: INFO os_vif [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15')
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:06:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:3f:e5:e9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:06:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updated VIF entry in instance network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:06:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61f06ef-06f5-41ad-93e1-1d81689c38e9 req-55aabbd8-9e4d-43a7-b59d-67c7501fbf18 service nova] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:35 user nova-compute[71323]: INFO nova.compute.manager [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Terminating instance
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-unplugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-unplugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-4fe091bb-fbea-423b-ba6d-db52144f46b6 req-311e07fc-4249-4f66-a746-fe300be88eb7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-unplugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:35 user nova-compute[71323]: WARNING nova.compute.manager [req-2518ee55-6991-4d7d-8cd2-64a5dc607033 req-b109a44b-fdb7-4e47-9967-2e4e0616c814 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state building and task_state spawning.
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Instance destroyed successfully.
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.objects.instance [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lazy-loading 'resources' on Instance uuid 1fb18db3-28b0-4886-a680-4c534d31cb6b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:48Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-934222543',display_name='tempest-ServersNegativeTestJSON-server-934222543',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-934222543',id=1,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f021342634dd4f16bf2720533070154c',ramdisk_id='',reservation_id='r-jc0pkunl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-675126189',owner_user_name='tempest-ServersNegativeTestJSON-675126189-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:58:07Z,user_data=None,user_id='0dbce21e767a4a61b9213e38523d247e',uuid=1fb18db3-28b0-4886-a680-4c534d31cb6b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converting VIF {"id": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "address": "fa:16:3e:d9:39:a8", "network": {"id": "47266f26-ea7c-404a-9098-264e4a1e0ab6", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1364599400-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f021342634dd4f16bf2720533070154c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf52f4b73-b4", "ovs_interfaceid": "f52f4b73-b4c4-4bc7-a565-1239a13770f1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG os_vif [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf52f4b73-b4, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:06:36 user nova-compute[71323]: INFO os_vif [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d9:39:a8,bridge_name='br-int',has_traffic_filtering=True,id=f52f4b73-b4c4-4bc7-a565-1239a13770f1,network=Network(47266f26-ea7c-404a-9098-264e4a1e0ab6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf52f4b73-b4')
Apr 14 03:06:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Deleting instance files /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b_del
Apr 14 03:06:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Deletion of /opt/stack/data/nova/instances/1fb18db3-28b0-4886-a680-4c534d31cb6b_del complete
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: INFO nova.compute.manager [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Took 1.02 seconds to destroy the instance on the hypervisor.
Apr 14 03:06:36 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:37 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Took 0.86 seconds to deallocate network for instance.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.304s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Deleted allocations for instance 1fb18db3-28b0-4886-a680-4c534d31cb6b
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86c853e2-ce1d-4b91-883e-0f466f40fb20 tempest-ServersNegativeTestJSON-675126189 tempest-ServersNegativeTestJSON-675126189-project-member] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.363s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:37 user nova-compute[71323]: WARNING nova.compute.manager [req-271acc11-8133-4b28-ab1d-de927e43316a req-9eb292c2-f824-45fa-b2b2-b23f458ac08f service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state building and task_state spawning.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:37 user nova-compute[71323]: WARNING nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state deleted and task_state None.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:37 user nova-compute[71323]: WARNING nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state deleted and task_state None.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:37 user nova-compute[71323]: WARNING nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state deleted and task_state None.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Acquiring lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] Lock "1fb18db3-28b0-4886-a680-4c534d31cb6b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] No waiting events found dispatching network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:37 user nova-compute[71323]: WARNING nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received unexpected event network-vif-plugged-f52f4b73-b4c4-4bc7-a565-1239a13770f1 for instance with vm_state deleted and task_state None.
Apr 14 03:06:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-2888c4fb-77c3-475f-82d6-9ba6a0efc436 req-3c52317d-048e-41bf-9fc5-7d6e705531b7 service nova] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Received event network-vif-deleted-f52f4b73-b4c4-4bc7-a565-1239a13770f1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441598.0113485, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Resumed (Lifecycle Event)
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance spawned successfully.
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441598.0124857, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Started (Lifecycle Event)
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Took 6.89 seconds to spawn the instance on the hypervisor.
Apr 14 03:06:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:38 user nova-compute[71323]: INFO nova.compute.manager [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Took 7.67 seconds to build instance.
Apr 14 03:06:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7dc9e926-ee72-4771-a3ec-acd2e5f6fd79 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.759s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:39 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'flavor' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.067s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:39 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441584.9557667, 2ca84990-b165-4d0b-a917-307cd8f96f28 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:06:39 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] VM Stopped (Lifecycle Event)
Apr 14 03:06:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-96430ff0-e42d-4340-b987-21789f0932e8 None None] [instance: 2ca84990-b165-4d0b-a917-307cd8f96f28] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:40 user nova-compute[71323]: INFO nova.compute.manager [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attaching volume 44f3cbc9-fd81-4185-aa74-0896739af983 to /dev/vdb
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.utils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b64764cc-4257-4f9f-a6f0-2a9e7afddabe]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[27d35e62-943c-48ad-92c3-aac588aa721a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[76c2638d-95d4-45bf-947c-8ca5f400a4a6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bc423be-5fe5-449d-8764-2a80d21c7695]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:40 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:06:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f127d3c8-0f81-4524-b788-eab30114a95c]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG os_brick.utils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:40 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating existing volume attachment record: 5c1e6dd8-018f-47a4-8b6b-3e57f0957faf {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:06:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:44 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:06:44 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:06:44 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:06:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0a66605-e8cd-4d4d-b559-51cf86c4b11b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:06:44 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[04b230c1-aef8-4576-9ddf-32f50ba089dc]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eec960e1-e75b-4abd-9262-cb92cc3fb905]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9905a7d6-469f-4596-9f05-7013c823b872]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4445e85f-81a0-4162-a282-ba8c7bf5b412]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'rCsYcqle'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cacd38f-635e-4b15-a5db-952173e6554a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c700a64a-89b3-4e6f-99da-fe83aeb9aedc]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login" returned: 0 in 0.057s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7514e49e-8c53-4612-adc3-b6721615b6b0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:06:45 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[77f42c48-2f47-4522-9fbb-c4f49bf154a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[422ffec8-a9db-444e-b8fa-09de240d7873]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:45 user nova-compute[71323]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:45 user nova-compute[71323]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:06:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.278s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('33', '-', '-', 0) found on session 20 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a1b0e84-c1e1-4904-9954-91abf27b7eca]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 20 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:06:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.328s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1330ms) {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'flavor' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:06:46 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:06:46 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:06:46 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:06:46 user nova-compute[71323]:   <serial>44f3cbc9-fd81-4185-aa74-0896739af983</serial>
Apr 14 03:06:46 user nova-compute[71323]: </disk>
Apr 14 03:06:46 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:af:bb:dd, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:06:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9df6022a-be13-4574-a908-5ad778bc77e7 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.659s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Stashing vm_state: active {{(pid=71323) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_requests' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:06:49 user nova-compute[71323]: INFO nova.compute.claims [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Claim successful on node user
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:06:49 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating resource usage from migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef
Apr 14 03:06:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:06:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.521s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.compute.manager [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Migrating
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "compute-rpcapi-router" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.115s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.compute.claims [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Claim successful on node user
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.compute.rpcapi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Automatically selected compute RPC version 6.2 from minimum service version 66
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "compute-rpcapi-router" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.compute.manager [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] instance snapshotting
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Beginning live snapshot process
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.428s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json -f qcow2" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:50 user nova-compute[71323]: INFO nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Booting with volume d7ac81e3-c104-4bb3-ada5-ecd6969601f0 at /dev/vda
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.policy [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5a311491e2924c49b01fd42458ec2fc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d531e7f7004ebaa529cb60de3849e7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json -f qcow2" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7dc312c0-3955-43ac-8fa4-47c9770836e1]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[076dd10a-0b7f-4ea6-a700-0c3200dd71db]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Starting migrate_disk_and_power_off {{(pid=71323) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0eab319e-bfd7-4882-b5b4-631a6b1792b0]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b069b2ca-3cb4-4249-8a8d-9cab9b830e9e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:50 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:06:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c60c593-fbae-47bd-b5bc-61cdbd52bcc5]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== get_connector_properties: return (158ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:50 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating existing volume attachment record: ea74a039-df1d-4c31-9fd6-1edcb23e777d {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.166s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48.delta 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441596.1047907, 1fb18db3-28b0-4886-a680-4c534d31cb6b => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:06:51 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] VM Stopped (Lifecycle Event)
Apr 14 03:06:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-34c7347e-e06c-4fea-9da6-528b4a3a6b0a None None] [instance: 1fb18db3-28b0-4886-a680-4c534d31cb6b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48.delta 1073741824" returned: 0 in 0.174s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 03:06:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:51 user nova-compute[71323]: WARNING nova.compute.manager [req-a6a94f1b-f13f-44c2-bffa-e22a370aed06 req-6ea2820d-649c-4790-9229-3df212f79c16 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:06:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:52 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance destroyed successfully.
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:06:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully created port: 2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap04b5c1f1-89, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:06:52 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 03:06:52 user nova-compute[71323]: INFO os_vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:52 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1cfa7c8-4f3c-4c04-8d59-37cd7f108a1f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:06:52 user nova-compute[71323]: No targets found.
Apr 14 03:06:52 user nova-compute[71323]: iSNS:
Apr 14 03:06:52 user nova-compute[71323]: No targets found.
Apr 14 03:06:52 user nova-compute[71323]: STATIC:
Apr 14 03:06:52 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:06:52 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:06:52 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:06:52 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:06:52 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:06:52 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:06:52 user nova-compute[71323]: FIRMWARE:
Apr 14 03:06:52 user nova-compute[71323]: No targets found.
Apr 14 03:06:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:06:52 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:06:52 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a87bea10-f21e-456a-89c0-ef8ae5529c44]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2fbce4b-bb2e-4e73-a234-8071968b22c4]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:52 user nova-compute[71323]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:06:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:06:52 user nova-compute[71323]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:06:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cdcad40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e605126-d166-4226-b00d-41e358644b84]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48.delta /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b978bb3-3a13-4903-8c13-6335d40e5ea8]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[188ad84e-1fc4-40d6-affa-dab9bbf08d9e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.054s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[823e241e-e2eb-4f3b-822c-1feeb464c188]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b914aef-62b7-48a3-86e7-e606334e7955]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout" returned: 0 in 0.224s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd187ca0-053f-40bb-94d9-a91f17bf3787]: (4, ('Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogout of [sid: 20, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:06:52 user nova-compute[71323]: Logout of [sid: 20, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:06:52 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8ce3fac-5c13-47c7-8502-8089e1315a96]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.588s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (589ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48.delta /opt/stack/data/nova/instances/snapshots/tmp20rq35jg/fe80d558a56344ffa9d0bd8bd37ceb48" returned: 0 in 0.560s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:53 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Snapshot extracted, beginning image upload
Apr 14 03:06:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_resize/disk /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully created port: 98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:53 user nova-compute[71323]: WARNING nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:53 user nova-compute[71323]: WARNING nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:53 user nova-compute[71323]: WARNING nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:53 user nova-compute[71323]: WARNING nova.compute.manager [req-b592fbf6-ebcc-4145-98fd-73252ef47058 req-bdf41638-205b-4c26-8cc7-358dc5e74b42 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:06:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "cp -r /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_resize/disk /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk" returned: 0 in 0.305s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_resize/disk.info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "cp -r /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_resize/disk.info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk.info" returned: 0 in 0.031s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully created port: dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 binding to destination host user is already ACTIVE {{(pid=71323) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully created port: 6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:55 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Snapshot image upload complete
Apr 14 03:06:55 user nova-compute[71323]: INFO nova.compute.manager [None req-6db44f17-0bd9-4355-9862-ddb738e88474 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Took 5.31 seconds to snapshot the instance on the hypervisor.
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully created port: 093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.utils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:06:55 user nova-compute[71323]: WARNING nova.compute.manager [req-f5fdcc89-78e5-4f74-aefb-8eda45e358ca req-7eb22812-5692-4b30-bdd3-5f0e1f4199ed service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_migrated.
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8d52ce7-ca4b-418a-b8ff-cc485124b344]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ed4afb8-6bbf-4208-aaa1-e1b24caed4b9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef22be87-2676-49dd-9a55-3bbb3a62d29d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[53ce046b-80fe-4c57-8180-618221e5cf97]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:55 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8616edbb-1d8f-42dc-8ece-0289f4435909]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.utils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:55 user nova-compute[71323]: INFO nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Booting with volume f9476602-ab81-47c8-aa31-ecc8e3c672ee at /dev/vdb
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[13f5a20c-1cee-4980-bb91-a4ddca00fb99]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5d6c1bc-b765-4444-9bb9-20d024c003e9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd584664-eeeb-46c0-b731-3dae5ccc3f28]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c326d4c-7115-417d-a0eb-3b62b3b80b1a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:06:55 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:06:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9218157c-c89d-45bc-be86-884ad81526b1]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating existing volume attachment record: e108a1db-4cd6-4bc4-9856-cab7507514d9 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:06:56 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: 2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-2e49423b-86d6-4b5a-b9cf-321ee37937c3. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:06:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: 0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-dbeb1c70-267b-4fde-b1a5-0e3567a11489. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb10713-e852-41fc-ac92-7aebdacebf2e req-1b119041-a34e-45db-afe2-06b100562793 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:06:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: 98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [{"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-bbb6cd58-ce7a-43ae-876d-51a888193f60" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-0a6ecbf1-8127-469a-92cc-5b4673e3db51. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:06:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-98ba28ec-b77c-4a23-b003-a4d29371cf95. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: 6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Starting finish_migration {{(pid=71323) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:07:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating image(s)
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'trusted_certs' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-dd090320-de86-40c8-bafb-515fa9c628e3. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: INFO nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Booting with volume e178f132-e302-4ee5-8ca6-45972e24754a at /dev/vdc
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4f5bdab-7823-449b-981d-2bd7bbd7b8ae]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[958a4104-4b4a-45ca-97d4-da52e04c09d4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9b36d4a-52eb-4fe6-bc81-471ba9f6bb27]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[95a3438d-8329-447c-8491-a5ce25107f29]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Cannot resize image /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:07:00 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb6d7071-5b56-4d4d-a7c2-f4ab118c19a2]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.utils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== get_connector_properties: return (146ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating existing volume attachment record: a8bb7005-4faf-4c15-926a-3d8c0ef7af87 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Ensure instance console log exists: /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start _get_guest_xml network_info=[{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '5118fd99-eb33-477a-930f-88dcb6f3179f', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4', 'attached_at': '2023-04-14T03:07:00.000000', 'detached_at': '', 'volume_id': '44f3cbc9-fd81-4185-aa74-0896739af983', 'serial': '44f3cbc9-fd81-4185-aa74-0896739af983'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:21Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-db6ed4a8-fc90-442d-85ca-83c3ec2ae6be req-08cb294d-56ae-4dea-bbed-edfa44a81172 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:00 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:07:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd07c1a4-e312-499f-b57d-8456f7151145]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:07:01 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad70578b-a9c2-4834-96a5-3a3e7ea72d66]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[35747f1f-e97f-49f8-b95a-a2b81471a53f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1a22196-cfa2-496b-87c8-c538f189278c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle" returned: 0 in 0.037s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc685cb0-a978-4e34-a53f-2309c5eb7c00]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'rCsYcqle'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[642d4f41-ba25-4296-827c-a3806f53b1a6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[552651b7-e130-4bdb-99a6-27fa5566a249]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login" returned: 0 in 0.057s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eff44949-dcec-4219-8cd4-add790e8181a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:07:01 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Successfully updated port: 093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e02d2d50-b7e3-4345-8fc6-4c090399e54f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[81b8d9d3-053e-4fe8-b952-113c056e9fb4]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:01 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:01 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.385s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('33', '-', '-', 0) found on session 21 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a41efbc4-10e7-4f83-873a-7182be076d7b]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 21 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-6691f178-7199-44c1-9dd7-049cf205185b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.738s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1740ms) {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:07:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:02 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7930MB free_disk=26.533832550048828GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:07:02 user nova-compute[71323]:   <uuid>6bb082bc-0c74-4149-9bd0-ceb09d7764d4</uuid>
Apr 14 03:07:02 user nova-compute[71323]:   <name>instance-00000005</name>
Apr 14 03:07:02 user nova-compute[71323]:   <memory>196608</memory>
Apr 14 03:07:02 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:07:02 user nova-compute[71323]:   <metadata>
Apr 14 03:07:02 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:02 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-1052167823</nova:name>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:07:00</nova:creationTime>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:flavor name="m1.micro">
Apr 14 03:07:02 user nova-compute[71323]:         <nova:memory>192</nova:memory>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:02 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:owner>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:07:02 user nova-compute[71323]:       </nova:owner>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:02 user nova-compute[71323]:       <nova:ports>
Apr 14 03:07:02 user nova-compute[71323]:         <nova:port uuid="04b5c1f1-8988-4a34-b97a-ad0cecd12b71">
Apr 14 03:07:02 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:07:02 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:02 user nova-compute[71323]:       </nova:ports>
Apr 14 03:07:02 user nova-compute[71323]:     </nova:instance>
Apr 14 03:07:02 user nova-compute[71323]:   </metadata>
Apr 14 03:07:02 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:07:02 user nova-compute[71323]:     <system>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="serial">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="uuid">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:07:02 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:07:02 user nova-compute[71323]:     </system>
Apr 14 03:07:02 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:02 user nova-compute[71323]:   <os>
Apr 14 03:07:02 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:07:02 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:07:02 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:07:02 user nova-compute[71323]:   </os>
Apr 14 03:07:02 user nova-compute[71323]:   <features>
Apr 14 03:07:02 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:02 user nova-compute[71323]:     <apic/>
Apr 14 03:07:02 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:07:02 user nova-compute[71323]:   </features>
Apr 14 03:07:02 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:07:02 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:07:02 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:07:02 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:07:02 user nova-compute[71323]:   </clock>
Apr 14 03:07:02 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:07:02 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:07:02 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:07:02 user nova-compute[71323]:   </cpu>
Apr 14 03:07:02 user nova-compute[71323]:   <devices>
Apr 14 03:07:02 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:07:02 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:07:02 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk"/>
Apr 14 03:07:02 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:07:02 user nova-compute[71323]:     </disk>
Apr 14 03:07:02 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:07:02 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:02 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:07:02 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:07:02 user nova-compute[71323]:       <serial>44f3cbc9-fd81-4185-aa74-0896739af983</serial>
Apr 14 03:07:02 user nova-compute[71323]:     </disk>
Apr 14 03:07:02 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:02 user nova-compute[71323]:       <mac address="fa:16:3e:af:bb:dd"/>
Apr 14 03:07:02 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:02 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:02 user nova-compute[71323]:       <target dev="tap04b5c1f1-89"/>
Apr 14 03:07:02 user nova-compute[71323]:     </interface>
Apr 14 03:07:02 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:07:02 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log" append="off"/>
Apr 14 03:07:02 user nova-compute[71323]:     </serial>
Apr 14 03:07:02 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:07:02 user nova-compute[71323]:     <video>
Apr 14 03:07:02 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:02 user nova-compute[71323]:     </video>
Apr 14 03:07:02 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:07:02 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:07:02 user nova-compute[71323]:     </rng>
Apr 14 03:07:02 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:07:02 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:07:02 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:07:02 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:02 user nova-compute[71323]:   </devices>
Apr 14 03:07:02 user nova-compute[71323]: </domain>
Apr 14 03:07:02 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:af:bb:dd"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG os_vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap04b5c1f1-89, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap04b5c1f1-89, col_values=(('external_ids', {'iface-id': '04b5c1f1-8988-4a34-b97a-ad0cecd12b71', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:af:bb:dd', 'vm-uuid': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:02 user nova-compute[71323]: INFO os_vif [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Applying migration context for instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 as it has an incoming, in-progress migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef. Migration status is post-migrating {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 03:07:02 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating resource usage from migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:af:bb:dd, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f970adb1-fe1b-4b58-83aa-45207a6211ef actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3e73add1-26be-47dd-adc5-506ab517544a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:07:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1856MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.477s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance network_info: |[{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:03 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updated VIF entry in instance network info cache for port 6691f178-7199-44c1-9dd7-049cf205185b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-093349f3-a544-487d-9927-f75c63ac2e9c. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:04 user nova-compute[71323]: WARNING nova.compute.manager [req-dbb81b53-87f7-47a0-97ed-ef7ddb69e3eb req-4cf429fd-895f-4e6c-ba57-a5986730cb7b service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state resize_finish.
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updated VIF entry in instance network info cache for port 093349f3-a544-487d-9927-f75c63ac2e9c. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.network.neutron [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a69e63e2-7c93-4910-8449-5f8eac31fd39 req-c7b9345f-6ade-4278-88eb-b481a4dea228 service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:07:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Creating image(s)
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Ensure instance console log exists: /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Start _get_guest_xml network_info=[{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'ea74a039-df1d-4c31-9fd6-1edcb23e777d', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dqJMIbXg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '29366fd0-eee1-4eb2-921a-c46b9677cc93', 'attached_at': '', 'detached_at': '', 'volume_id': 'd7ac81e3-c104-4bb3-ada5-ecd6969601f0', 'serial': 'd7ac81e3-c104-4bb3-ada5-ecd6969601f0'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None}, {'device_type': 'disk', 'attachment_id': 'e108a1db-4cd6-4bc4-9856-cab7507514d9', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PhzIAFeq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '29366fd0-eee1-4eb2-921a-c46b9677cc93', 'attached_at': '', 'detached_at': '', 'volume_id': 'f9476602-ab81-47c8-aa31-ecc8e3c672ee', 'serial': 'f9476602-ab81-47c8-aa31-ecc8e3c672ee'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': 1, 'volume_type': None}, {'device_type': 'disk', 'attachment_id': 'a8bb7005-4faf-4c15-926a-3d8c0ef7af87', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'akyODBJC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '29366fd0-eee1-4eb2-921a-c46b9677cc93', 'attached_at': '', 'detached_at': '', 'volume_id': 'e178f132-e302-4ee5-8ca6-45972e24754a', 'serial': 'e178f132-e302-4ee5-8ca6-45972e24754a'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdc', 'delete_on_termination': False, 'boot_index': 2, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:07:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dqJMIbXg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:05 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb5cd261-7efe-498f-b586-5c9ea5348533]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:07:05 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7debead9-daff-4066-bf83-373654573faf]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0] added
Apr 14 03:07:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[81bb236f-f609-4271-97f8-181817b332fc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e59d09c0-757e-408e-9496-e09c78142a8f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dqJMIbXg {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dqJMIbXg" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3db80d2-f48b-4d9c-950f-180703a9cc69]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'dqJMIbXg'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[84cefd8f-3f8d-438e-88ff-6f400dfb0aa5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[109cc5b8-3b2a-4c0f-88e2-49bd493f6f94]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:05 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:05 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d06447dd-d413-41e4-bdf8-980ee888dd9b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260]
Apr 14 03:07:06 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b9be52f-3125-4ee9-a18d-8771ea07c92f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c9478c6-9b56-49fe-b8f2-35ad6f9ad187]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:06 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:06 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:06 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:06 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:06 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.283s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] HCTL ('34', '-', '-', 0) found on session 22 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7671df46-3c89-49bf-949f-033bb431388b]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Searching for a device in session 22 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dqJMIbXg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441626.3624341, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Resumed (Lifecycle Event)
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:07:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance running successfully.
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Failed to set time: agent not configured {{(pid=71323) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8c16f637-7eb1-4f78-9a98-2ac42b219ca2 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] finish_migration finished successfully. {{(pid=71323) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:07:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441626.3629522, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Started (Lifecycle Event)
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:07:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:06 user nova-compute[71323]: WARNING nova.compute.manager [req-d276b39b-d708-4224-a9e7-f859afdd6033 req-c95e742b-8346-4376-a48e-aced4446bad5 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state resized and task_state None.
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.315s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== connect_volume: return (1316ms) {'type': 'block', 'scsi_wwn': '26635323338656363', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26635323338656363', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PhzIAFeq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:07 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:07 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:07 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a441581d-c2ac-4cc0-a37f-d332ce6a4d75]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:07:07 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d15d65ce-74bf-4fb8-afcb-f23677c8b324]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee] added
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae896e26-0a94-432d-a92d-ad78862ec2e4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e6d3a7d-26fe-449c-9772-eadc40f19391]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PhzIAFeq {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.username -v PhzIAFeq" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa084fb0-95cd-47b3-8ea1-884c75a379de]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PhzIAFeq'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9922d376-2f63-496b-b65c-f320e3da2613]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5abd2fdd-14f8-4d01-aa45-9a919a6a5ea3]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3587f0f0-b702-4e13-adac-719e5ca9f86b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260]
Apr 14 03:07:07 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3a672db-4436-42bf-86ec-97373fb693f6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[98c1cca4-4627-403f-a7ce-adbda1cdc009]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:07 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.300s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] HCTL ('35', '-', '-', 0) found on session 23 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aac78138-a223-4af1-a1b0-5e25439995c6]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Searching for a device in session 23 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:08 user nova-compute[71323]: INFO nova.compute.manager [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Terminating instance
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Searching for a device in session 23 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PhzIAFeq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.331s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23336323238646235', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23336323238646235', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'akyODBJC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:08 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:08 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:08 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4aed41f-d9f8-4087-92e7-752ae0a58fe4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:07:08 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4da433eb-6ce3-4d1d-8a6f-73f54c327ddb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a] added
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a1585b0-aca9-4c6d-9e87-2a29bbcb607a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-unplugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] No waiting events found dispatching network-vif-unplugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb719e03-5687-4b4f-977a-cf406fb0037f req-f74864be-1e0d-48ab-909c-307ff474523a service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-unplugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.077s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9b456d5-97af-4662-8810-fbbdc98df8fe]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v akyODBJC {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v akyODBJC" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[01105907-6167-4f13-beaa-5123f71e9a24]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'akyODBJC'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4be60b8-bf1f-4abd-adc7-9c8831d3345c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18416daa-de86-423d-9781-82ee2464dada]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7cd523f-8dcc-479a-89be-5cb3de44ed12]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260]
Apr 14 03:07:08 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ed7fcda-b3f4-44bc-824b-ba2774085f95]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7319c1c-7c35-4f08-b0b9-03550ed3f8c0]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:08 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.382s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] HCTL ('36', '-', '-', 0) found on session 24 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[06dea4aa-6c3d-4fcd-a89f-eb9c98f0966a]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Searching for a device in session 24 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'akyODBJC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:07:09 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Instance destroyed successfully.
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid f970adb1-fe1b-4b58-83aa-45207a6211ef {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:02:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1182227426',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1182227426',id=15,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-vngr2krk',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:02:42Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=f970adb1-fe1b-4b58-83aa-45207a6211ef,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "address": "fa:16:3e:0c:70:d3", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape51353cf-a5", "ovs_interfaceid": "e51353cf-a5e9-41e4-9b6e-4a022984af76", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_vif [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape51353cf-a5, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:09 user nova-compute[71323]: INFO os_vif [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0c:70:d3,bridge_name='br-int',has_traffic_filtering=True,id=e51353cf-a5e9-41e4-9b6e-4a022984af76,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape51353cf-a5')
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YcTtkeKx', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:09 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.417s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== connect_volume: return (1418ms) {'type': 'block', 'scsi_wwn': '26536346338343035', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26536346338343035', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.843s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f144d71-dcba-4a99-a77f-2350fdd1fe49]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:07:09 user nova-compute[71323]: No targets found.
Apr 14 03:07:09 user nova-compute[71323]: iSNS:
Apr 14 03:07:09 user nova-compute[71323]: No targets found.
Apr 14 03:07:09 user nova-compute[71323]: STATIC:
Apr 14 03:07:09 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e
Apr 14 03:07:09 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:09 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:09 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0
Apr 14 03:07:09 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:09 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:09 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a
Apr 14 03:07:09 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:09 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:09 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:07:09 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:09 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:09 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:07:09 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:09 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:09 user nova-compute[71323]: FIRMWARE:
Apr 14 03:07:09 user nova-compute[71323]: No targets found.
Apr 14 03:07:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:07:09 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:07:09 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lazy-loading 'pci_devices' on Instance uuid 29366fd0-eee1-4eb2-921a-c46b9677cc93 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c17fdf6-6a8b-468e-b661-a27c7c0f465c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:07:10 user nova-compute[71323]:   <uuid>29366fd0-eee1-4eb2-921a-c46b9677cc93</uuid>
Apr 14 03:07:10 user nova-compute[71323]:   <name>instance-00000015</name>
Apr 14 03:07:10 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:07:10 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:07:10 user nova-compute[71323]:   <metadata>
Apr 14 03:07:10 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:10 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:name>tempest-device-tagging-server-875179344</nova:name>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:07:05</nova:creationTime>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:07:10 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:10 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:owner>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:user uuid="5a311491e2924c49b01fd42458ec2fc2">tempest-TaggedBootDevicesTest_v242-641324897-project-member</nova:user>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:project uuid="f1d531e7f7004ebaa529cb60de3849e7">tempest-TaggedBootDevicesTest_v242-641324897</nova:project>
Apr 14 03:07:10 user nova-compute[71323]:       </nova:owner>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:10 user nova-compute[71323]:       <nova:ports>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="2e49423b-86d6-4b5a-b9cf-321ee37937c3">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="dbeb1c70-267b-4fde-b1a5-0e3567a11489">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.189" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="0a6ecbf1-8127-469a-92cc-5b4673e3db51">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.110" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="98ba28ec-b77c-4a23-b003-a4d29371cf95">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.113" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="dd090320-de86-40c8-bafb-515fa9c628e3">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.1.1.155" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="6691f178-7199-44c1-9dd7-049cf205185b">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:         <nova:port uuid="093349f3-a544-487d-9927-f75c63ac2e9c">
Apr 14 03:07:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Apr 14 03:07:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:07:10 user nova-compute[71323]:       </nova:ports>
Apr 14 03:07:10 user nova-compute[71323]:     </nova:instance>
Apr 14 03:07:10 user nova-compute[71323]:   </metadata>
Apr 14 03:07:10 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:07:10 user nova-compute[71323]:     <system>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="serial">29366fd0-eee1-4eb2-921a-c46b9677cc93</entry>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="uuid">29366fd0-eee1-4eb2-921a-c46b9677cc93</entry>
Apr 14 03:07:10 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:07:10 user nova-compute[71323]:     </system>
Apr 14 03:07:10 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:10 user nova-compute[71323]:   <os>
Apr 14 03:07:10 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:07:10 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:07:10 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:07:10 user nova-compute[71323]:   </os>
Apr 14 03:07:10 user nova-compute[71323]:   <features>
Apr 14 03:07:10 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:10 user nova-compute[71323]:     <apic/>
Apr 14 03:07:10 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:07:10 user nova-compute[71323]:   </features>
Apr 14 03:07:10 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:07:10 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:07:10 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:07:10 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:07:10 user nova-compute[71323]:   </clock>
Apr 14 03:07:10 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:07:10 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:07:10 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:07:10 user nova-compute[71323]:   </cpu>
Apr 14 03:07:10 user nova-compute[71323]:   <devices>
Apr 14 03:07:10 user nova-compute[71323]:     <disk type="file" device="cdrom">
Apr 14 03:07:10 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none"/>
Apr 14 03:07:10 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/disk.config"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="hda" bus="ide"/>
Apr 14 03:07:10 user nova-compute[71323]:     </disk>
Apr 14 03:07:10 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:07:10 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:10 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <serial>d7ac81e3-c104-4bb3-ada5-ecd6969601f0</serial>
Apr 14 03:07:10 user nova-compute[71323]:     </disk>
Apr 14 03:07:10 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:07:10 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:10 user nova-compute[71323]:       <source dev="/dev/sdd"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <serial>f9476602-ab81-47c8-aa31-ecc8e3c672ee</serial>
Apr 14 03:07:10 user nova-compute[71323]:     </disk>
Apr 14 03:07:10 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:07:10 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:10 user nova-compute[71323]:       <source dev="/dev/sde"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="vdc" bus="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <serial>e178f132-e302-4ee5-8ca6-45972e24754a</serial>
Apr 14 03:07:10 user nova-compute[71323]:     </disk>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:5a:7c:e0"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tap2e49423b-86"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:10:a2:b1"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tapdbeb1c70-26"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:3c:51:36"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tap0a6ecbf1-81"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:ae:42:b8"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tap98ba28ec-b7"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:61:e3:f3"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tapdd090320-de"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:48:b5:68"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tap6691f178-71"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:07:10 user nova-compute[71323]:       <mac address="fa:16:3e:63:33:1a"/>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:07:10 user nova-compute[71323]:       <target dev="tap093349f3-a5"/>
Apr 14 03:07:10 user nova-compute[71323]:     </interface>
Apr 14 03:07:10 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:07:10 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/console.log" append="off"/>
Apr 14 03:07:10 user nova-compute[71323]:     </serial>
Apr 14 03:07:10 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:07:10 user nova-compute[71323]:     <video>
Apr 14 03:07:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:07:10 user nova-compute[71323]:     </video>
Apr 14 03:07:10 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:07:10 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:07:10 user nova-compute[71323]:     </rng>
Apr 14 03:07:10 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:07:10 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:07:10 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:07:10 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:10 user nova-compute[71323]:   </devices>
Apr 14 03:07:10 user nova-compute[71323]: </domain>
Apr 14 03:07:10 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2e49423b-86, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2e49423b-86, col_values=(('external_ids', {'iface-id': '2e49423b-86d6-4b5a-b9cf-321ee37937c3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5a:7c:e0', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc1c5580-8e18-40de-80e1-1230244a1b79]: (4, ('tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:10 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1a37e830>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e'): ({'sdg'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdg {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdbeb1c70-26, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdbeb1c70-26, col_values=(('external_ids', {'iface-id': 'dbeb1c70-267b-4fde-b1a5-0e3567a11489', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:10:a2:b1', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3208b99f-374b-444d-865a-e6271bc31195]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0a6ecbf1-81, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap0a6ecbf1-81, col_values=(('external_ids', {'iface-id': '0a6ecbf1-8127-469a-92cc-5b4673e3db51', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3c:51:36', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b89dd386-f3e3-40b5-ba0f-f3c8ee427191]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdg {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c5e922c-2d39-4213-ae4a-42cf89f482f9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap98ba28ec-b7, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap98ba28ec-b7, col_values=(('external_ids', {'iface-id': '98ba28ec-b77c-4a23-b003-a4d29371cf95', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ae:42:b8', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.050s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6283d33-0a62-4d0d-b54b-d04b3836f243]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdg have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdd090320-de, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapdd090320-de, col_values=(('external_ids', {'iface-id': 'dd090320-de86-40c8-bafb-515fa9c628e3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:61:e3:f3', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a6f4e5d-d081-4205-9349-a1ba77e1c794]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6691f178-71, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6691f178-71, col_values=(('external_ids', {'iface-id': '6691f178-7199-44c1-9dd7-049cf205185b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:48:b5:68', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2037957b-f4c1-466a-90e3-161423628a0e]: (4, ('Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260]\nLogout of [sid: 14, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260]
Apr 14 03:07:10 user nova-compute[71323]: Logout of [sid: 14, target: iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:37bd0d93-b922-426c-8986-d86e886da69e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7e8861a-89bb-400a-baef-bcc68cfb3672]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.381s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (1224ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Deleting instance files /opt/stack/data/nova/instances/f970adb1-fe1b-4b58-83aa-45207a6211ef_del
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Deletion of /opt/stack/data/nova/instances/f970adb1-fe1b-4b58-83aa-45207a6211ef_del complete
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap093349f3-a5, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap093349f3-a5, col_values=(('external_ids', {'iface-id': '093349f3-a544-487d-9927-f75c63ac2e9c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:63:33:1a', 'vm-uuid': '29366fd0-eee1-4eb2-921a-c46b9677cc93'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.compute.manager [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Took 2.06 seconds to destroy the instance on the hypervisor.
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:10 user nova-compute[71323]: INFO os_vif [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5')
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] No BDM found with device name hda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] No VIF found with MAC fa:16:3e:5a:7c:e0, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] No VIF found with MAC fa:16:3e:61:e3:f3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Using config drive
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Creating config drive at /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/disk.config
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpw6fuw0en {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpw6fuw0en" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] Acquiring lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] No waiting events found dispatching network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:10 user nova-compute[71323]: WARNING nova.compute.manager [req-2f0aafd2-0775-4028-9cb2-53fae0f903a8 req-6521c031-9435-42db-9e45-3c862e836852 service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received unexpected event network-vif-plugged-e51353cf-a5e9-41e4-9b6e-4a022984af76 for instance with vm_state active and task_state deleting.
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-5712b1dd-16c6-4036-aed2-b47d9e109a7a req-8ca444cf-3aa6-4bf7-adea-09cf4588d09b service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Received event network-vif-deleted-e51353cf-a5e9-41e4-9b6e-4a022984af76 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.compute.manager [req-5712b1dd-16c6-4036-aed2-b47d9e109a7a req-8ca444cf-3aa6-4bf7-adea-09cf4588d09b service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Neutron deleted interface e51353cf-a5e9-41e4-9b6e-4a022984af76; detaching it from the instance and deleting it from the info cache
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-5712b1dd-16c6-4036-aed2-b47d9e109a7a req-8ca444cf-3aa6-4bf7-adea-09cf4588d09b service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:10 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Took 0.48 seconds to deallocate network for instance.
Apr 14 03:07:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-5712b1dd-16c6-4036-aed2-b47d9e109a7a req-8ca444cf-3aa6-4bf7-adea-09cf4588d09b service nova] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Detach interface failed, port_id=e51353cf-a5e9-41e4-9b6e-4a022984af76, reason: Instance f970adb1-fe1b-4b58-83aa-45207a6211ef could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:07:11 user nova-compute[71323]: INFO nova.compute.manager [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Took 0.86 seconds to detach 1 volumes for instance.
Apr 14 03:07:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Deleting volume: 37bd0d93-b922-426c-8986-d86e886da69e {{(pid=71323) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 03:07:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.523s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:12 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance f970adb1-fe1b-4b58-83aa-45207a6211ef
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:12 user nova-compute[71323]: WARNING nova.compute.manager [req-d937b6c8-ab63-4a0e-aa67-f355febe5c4e req-6a603496-874d-4ddb-9680-2e43c26de1ca service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 for instance with vm_state building and task_state spawning.
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c672aa91-49fa-4708-ae02-e93e69b845f4 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "f970adb1-fe1b-4b58-83aa-45207a6211ef" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.338s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:13 user nova-compute[71323]: WARNING nova.compute.manager [req-aca1d7c3-6435-48a7-a04d-a7c61554b458 req-2ba18f84-cd99-4c88-93f6-2add9678ef2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 for instance with vm_state building and task_state spawning.
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:14 user nova-compute[71323]: WARNING nova.compute.manager [req-1a2276e9-0449-4130-90ba-0aca0d59e42a req-3ebf4823-245d-4517-8407-685a8d79896e service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b for instance with vm_state building and task_state spawning.
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:15 user nova-compute[71323]: WARNING nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 for instance with vm_state building and task_state spawning.
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:15 user nova-compute[71323]: WARNING nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 for instance with vm_state building and task_state spawning.
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:15 user nova-compute[71323]: WARNING nova.compute.manager [req-f04392ca-acf2-4a89-bb2b-ffb60a72b983 req-2aba8e91-9855-4382-9b77-8cdf9929bb5d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 for instance with vm_state building and task_state spawning.
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-changed-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing instance network info cache due to event network-changed-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG nova.network.neutron [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing network info cache for port fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-None" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-None" acquired by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.network.neutron [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updated VIF entry in instance network info cache for port fb9a2be7-8d62-4528-a23c-bfdf995bd6dc. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.network.neutron [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c71f951-58fb-4db0-a818-ffe2a0182e7c req-da48924e-ccdd-4aab-a3cd-f98e3e9fd657 service nova] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'pci_requests' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.policy [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '17999fdd4eb449d49d74b4de20f6614a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6a991a4fd13b4a819b24479a341ba08c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441636.7973404, 29366fd0-eee1-4eb2-921a-c46b9677cc93 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] VM Resumed (Lifecycle Event)
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:16 user nova-compute[71323]: WARNING nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c for instance with vm_state building and task_state spawning.
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:16 user nova-compute[71323]: WARNING nova.compute.manager [req-a24f90b1-52e5-424b-9878-6a4ec987741f req-a6f4a2f9-9139-4bb4-8050-c8710dd7bc2d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c for instance with vm_state building and task_state spawning.
Apr 14 03:07:16 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance spawned successfully.
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:07:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441636.802083, 29366fd0-eee1-4eb2-921a-c46b9677cc93 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] VM Started (Lifecycle Event)
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:07:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:07:17 user nova-compute[71323]: INFO nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Took 11.27 seconds to spawn the instance on the hypervisor.
Apr 14 03:07:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:17 user nova-compute[71323]: INFO nova.compute.manager [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Took 27.16 seconds to build instance.
Apr 14 03:07:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f19b7c04-aa39-4c1a-bb81-db34f9afb584 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 27.267s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Successfully created port: bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Successfully updated port: bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.compute.manager [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-changed-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.compute.manager [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing instance network info cache due to event network-changed-bf026195-619f-4db1-a57a-1b3c0c83eb50. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.neutron [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Refreshing network info cache for port bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG os_vif [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbf026195-61, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapbf026195-61, col_values=(('external_ids', {'iface-id': 'bf026195-619f-4db1-a57a-1b3c0c83eb50', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fc:c1:87', 'vm-uuid': '3e73add1-26be-47dd-adc5-506ab517544a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: INFO os_vif [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61')
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] attach device xml: <interface type="ethernet">
Apr 14 03:07:19 user nova-compute[71323]:   <mac address="fa:16:3e:fc:c1:87"/>
Apr 14 03:07:19 user nova-compute[71323]:   <model type="virtio"/>
Apr 14 03:07:19 user nova-compute[71323]:   <mtu size="1442"/>
Apr 14 03:07:19 user nova-compute[71323]:   <target dev="tapbf026195-61"/>
Apr 14 03:07:19 user nova-compute[71323]: </interface>
Apr 14 03:07:19 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name hda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No VIF found with MAC fa:16:3e:e7:c3:de, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:19 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:19</nova:creationTime>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:19 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:19 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:19 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:19 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:19 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:19 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:19 user nova-compute[71323]:     <nova:port uuid="bf026195-619f-4db1-a57a-1b3c0c83eb50">
Apr 14 03:07:19 user nova-compute[71323]:       <nova:ip type="fixed" address="10.10.10.137" ipVersion="4"/>
Apr 14 03:07:19 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:19 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:19 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:19 user nova-compute[71323]:  {{(pid=71323) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8bc3c1aa-a8c1-4a67-81a9-5376d44f11d8 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-None" "released" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: held 3.460s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.network.neutron [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updated VIF entry in instance network info cache for port bf026195-619f-4db1-a57a-1b3c0c83eb50. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.network.neutron [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c55d7194-6b60-4e54-a6cb-f0fe09e5ee6d req-7716d8f5-2105-4da8-8722-6d8328dd3906 service nova] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:19 user nova-compute[71323]: WARNING nova.compute.manager [req-0c190eed-cc31-4146-937c-31291485c490 req-0b00cffe-566a-4d39-a879-44da5911928f service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 for instance with vm_state active and task_state None.
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:20 user nova-compute[71323]: INFO nova.compute.manager [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Attaching volume 5106df68-dc27-4e16-9ea7-9c77a3d4ca23 to /dev/vdb
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b80f3c39-d066-48d2-b93f-ad5b75078741]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad029717-23fb-4ded-a929-a17174081607]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d13b11f-0e5e-4e99-8008-98331ce0764d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb69a220-fdcc-4840-95f5-0b7f6edf8ec4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:07:20 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:07:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b925463e-fac9-48fe-a493-578243e5729c]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:20 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating existing volume attachment record: 2f658a02-7534-4f72-a836-304e187b4f8a {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:21 user nova-compute[71323]: WARNING nova.compute.manager [req-99904da6-368c-42bf-a012-5c9f9e6fbe44 req-8bb60d8a-d4bb-4719-bff3-ef56de26aaf5 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 for instance with vm_state active and task_state None.
Apr 14 03:07:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441629.0465007, f970adb1-fe1b-4b58-83aa-45207a6211ef => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:24 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] VM Stopped (Lifecycle Event)
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2f23ad81-0466-4e81-a553-1c2a8d61eb51 None None] [instance: f970adb1-fe1b-4b58-83aa-45207a6211ef] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:24 user nova-compute[71323]: INFO nova.compute.manager [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Terminating instance
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-f451386b-eb51-4757-bd85-95c6155582fc req-29167ba6-6e24-4323-9bff-2266fa3b2424 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Instance destroyed successfully.
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'resources' on Instance uuid bbb6cd58-ce7a-43ae-876d-51a888193f60 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1854200462',display_name='tempest-AttachVolumeTestJSON-server-1854200462',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1854200462',id=2,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGeBO3Puf45GsCORiHAZ27NGwlcWdsEQ3UWkZa0RxlsM/TNUaZ2QpW5Vj4DhEQilp60b+VAsZqstaUKI2WmSuZE1HgEzm2SMgB1a286kwGwkyhWrq2SJzF9Whk3OEHbnug==',key_name='tempest-keypair-1156063861',keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:58:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-0k4tf1bc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=bbb6cd58-ce7a-43ae-876d-51a888193f60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "b85a839f-973d-415e-94b6-cdb480a8b89d", "address": "fa:16:3e:f7:89:a6", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.70", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb85a839f-97", "ovs_interfaceid": "b85a839f-973d-415e-94b6-cdb480a8b89d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG os_vif [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb85a839f-97, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:24 user nova-compute[71323]: INFO os_vif [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f7:89:a6,bridge_name='br-int',has_traffic_filtering=True,id=b85a839f-973d-415e-94b6-cdb480a8b89d,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb85a839f-97')
Apr 14 03:07:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Deleting instance files /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60_del
Apr 14 03:07:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Deletion of /opt/stack/data/nova/instances/bbb6cd58-ce7a-43ae-876d-51a888193f60_del complete
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: INFO nova.compute.manager [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Took 1.05 seconds to destroy the instance on the hypervisor.
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qSwoCLQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:25 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260" returned: 21 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ee980b4-7dc0-4187-b807-40dc396f8f33]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:07:25 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b10a792f-bc71-4f1b-ad2f-271fd352b157]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23] added
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f257d3a3-e28c-4b64-8370-62807428b126]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[059f4c79-d74e-4ad6-b554-e7d08956fc7b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qSwoCLQN {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v qSwoCLQN" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[701e44ca-d94a-4a6e-8ebc-09e9d667390d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'qSwoCLQN'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d01966c9-bf03-4c3d-9940-c206f7b9be2b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[066c3736-6d8f-46f0-8488-8d0a15441b26]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --login" returned: 0 in 0.076s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[89864d71-9398-4945-9a6d-004d91357320]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260]
Apr 14 03:07:25 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfcb5d81-e45b-4600-af7d-81268baf7cc2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[560a2907-53cf-4d3a-b98d-ce84dcf98e7a]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:25 user nova-compute[71323]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)
Apr 14 03:07:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.334s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] HCTL ('37', '-', '-', 0) found on session 25 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c858c5ac-c6eb-43d7-95eb-5f4c988f8f1f]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Searching for a device in session 25 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qSwoCLQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state deleting.
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state deleting.
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state deleting.
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-unplugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Acquiring lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] No waiting events found dispatching network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:26 user nova-compute[71323]: WARNING nova.compute.manager [req-e0b39718-6c20-47dc-80c2-f5e218177f7b req-49161faf-97fa-4fcd-a056-e4cf84ae1dc4 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received unexpected event network-vif-plugged-b85a839f-973d-415e-94b6-cdb480a8b89d for instance with vm_state active and task_state deleting.
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:26 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Took 1.51 seconds to deallocate network for instance.
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-d548eec4-fd0d-46bb-bc03-0af3bcd075ec req-4d3a5579-6295-4ba7-adc8-63238963d712 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Received event network-vif-deleted-b85a839f-973d-415e-94b6-cdb480a8b89d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:26 user nova-compute[71323]: INFO nova.compute.manager [req-d548eec4-fd0d-46bb-bc03-0af3bcd075ec req-4d3a5579-6295-4ba7-adc8-63238963d712 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Neutron deleted interface b85a839f-973d-415e-94b6-cdb480a8b89d; detaching it from the instance and deleting it from the info cache
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.network.neutron [req-d548eec4-fd0d-46bb-bc03-0af3bcd075ec req-4d3a5579-6295-4ba7-adc8-63238963d712 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.397s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] <== connect_volume: return (1398ms) {'type': 'block', 'scsi_wwn': '26430636332343165', 'path': '/dev/sdf'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26430636332343165', 'path': '/dev/sdf'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-d548eec4-fd0d-46bb-bc03-0af3bcd075ec req-4d3a5579-6295-4ba7-adc8-63238963d712 service nova] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Detach interface failed, port_id=b85a839f-973d-415e-94b6-cdb480a8b89d, reason: Instance bbb6cd58-ce7a-43ae-876d-51a888193f60 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:07:26 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:26 user nova-compute[71323]:   <source dev="/dev/sdf"/>
Apr 14 03:07:26 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:07:26 user nova-compute[71323]:   <serial>5106df68-dc27-4e16-9ea7-9c77a3d4ca23</serial>
Apr 14 03:07:26 user nova-compute[71323]: </disk>
Apr 14 03:07:26 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.393s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:27 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Deleted allocations for instance bbb6cd58-ce7a-43ae-876d-51a888193f60
Apr 14 03:07:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No BDM found with device name hda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] No VIF found with MAC fa:16:3e:e7:c3:de, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3f999d18-258d-4bcc-bbfa-72c079b4a049 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "bbb6cd58-ce7a-43ae-876d-51a888193f60" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.237s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b225a6ca-535b-4552-bead-7b02332b05ee tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.281s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:28 user nova-compute[71323]: INFO nova.compute.manager [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Detaching volume 5106df68-dc27-4e16-9ea7-9c77a3d4ca23
Apr 14 03:07:28 user nova-compute[71323]: INFO nova.virt.block_device [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Attempting to driver detach volume 5106df68-dc27-4e16-9ea7-9c77a3d4ca23 from mountpoint /dev/vdb
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Attempting to detach device vdb from instance 3e73add1-26be-47dd-adc5-506ab517544a from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:07:28 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:28 user nova-compute[71323]:   <source dev="/dev/sdf"/>
Apr 14 03:07:28 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:07:28 user nova-compute[71323]:   <serial>5106df68-dc27-4e16-9ea7-9c77a3d4ca23</serial>
Apr 14 03:07:28 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 03:07:28 user nova-compute[71323]: </disk>
Apr 14 03:07:28 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:07:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully detached device vdb from instance 3e73add1-26be-47dd-adc5-506ab517544a from the persistent domain config.
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3e73add1-26be-47dd-adc5-506ab517544a from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:07:28 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:07:28 user nova-compute[71323]:   <source dev="/dev/sdf"/>
Apr 14 03:07:28 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:07:28 user nova-compute[71323]:   <serial>5106df68-dc27-4e16-9ea7-9c77a3d4ca23</serial>
Apr 14 03:07:28 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 03:07:28 user nova-compute[71323]: </disk>
Apr 14 03:07:28 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441648.90237, 3e73add1-26be-47dd-adc5-506ab517544a => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:07:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully detached device vdb from instance 3e73add1-26be-47dd-adc5-506ab517544a from the live domain config.
Apr 14 03:07:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'qSwoCLQN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:07:28 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[04e59ad4-2930-4565-b6a8-93aa778cbe8e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:07:28 user nova-compute[71323]: No targets found.
Apr 14 03:07:28 user nova-compute[71323]: iSNS:
Apr 14 03:07:28 user nova-compute[71323]: No targets found.
Apr 14 03:07:28 user nova-compute[71323]: STATIC:
Apr 14 03:07:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0
Apr 14 03:07:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a
Apr 14 03:07:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23
Apr 14 03:07:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:07:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:28 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:07:28 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:07:28 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:07:28 user nova-compute[71323]: FIRMWARE:
Apr 14 03:07:28 user nova-compute[71323]: No targets found.
Apr 14 03:07:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:07:28 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:07:28 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0f2fafd-5cfd-4ec8-8628-49791677018c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2463bd37-037b-4fd0-8230-789353856c77]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:07:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:07:28 user nova-compute[71323]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 (non-flash)
Apr 14 03:07:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ac1fa0a60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23'): ({'sdf'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Removing single pathed devices sdf {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[006818a7-9f0a-4eb5-8c17-3f5774f510df]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2acbd09b-4ba0-48a3-bd9c-851f8ff68dd9]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Flushing IO for device /dev/sdf {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a81ef572-fa2f-4ae7-9868-0a56e2fa32b7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f7043b1-84d3-4a0c-908c-49427de1c92c]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d96ae57c-b7ca-4a7a-b150-8b0b901dcbf4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0683d7e-b2e8-49a7-83cb-94cfcfe85955]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260]
Apr 14 03:07:29 user nova-compute[71323]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23, portal: 172.16.0.220,3260] successful.
Apr 14 03:07:29 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5106df68-dc27-4e16-9ea7-9c77a3d4ca23 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e20c08e-2a05-43b7-844d-f20679734574]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.341s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] <== disconnect_volume: return (342ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:07:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fb5db196-997f-4788-9f3f-be8b1c8eb506 tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.898s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-bf026195-619f-4db1-a57a-1b3c0c83eb50" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-bf026195-619f-4db1-a57a-1b3c0c83eb50" acquired by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Attempting to detach device tapbf026195-61 from instance 3e73add1-26be-47dd-adc5-506ab517544a from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] detach device xml: <interface type="ethernet">
Apr 14 03:07:30 user nova-compute[71323]:   <mac address="fa:16:3e:fc:c1:87"/>
Apr 14 03:07:30 user nova-compute[71323]:   <model type="virtio"/>
Apr 14 03:07:30 user nova-compute[71323]:   <mtu size="1442"/>
Apr 14 03:07:30 user nova-compute[71323]:   <target dev="tapbf026195-61"/>
Apr 14 03:07:30 user nova-compute[71323]: </interface>
Apr 14 03:07:30 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface>not found in domain: <domain type='kvm' id='24'>
Apr 14 03:07:30 user nova-compute[71323]:   <name>instance-00000013</name>
Apr 14 03:07:30 user nova-compute[71323]:   <uuid>3e73add1-26be-47dd-adc5-506ab517544a</uuid>
Apr 14 03:07:30 user nova-compute[71323]:   <metadata>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:30 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:19</nova:creationTime>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:30 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:30 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:30 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:port uuid="bf026195-619f-4db1-a57a-1b3c0c83eb50">
Apr 14 03:07:30 user nova-compute[71323]:       <nova:ip type="fixed" address="10.10.10.137" ipVersion="4"/>
Apr 14 03:07:30 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:30 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:30 user nova-compute[71323]:   </metadata>
Apr 14 03:07:30 user nova-compute[71323]:   <memory unit='KiB'>131072</memory>
Apr 14 03:07:30 user nova-compute[71323]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 03:07:30 user nova-compute[71323]:   <vcpu placement='static'>1</vcpu>
Apr 14 03:07:30 user nova-compute[71323]:   <resource>
Apr 14 03:07:30 user nova-compute[71323]:     <partition>/machine</partition>
Apr 14 03:07:30 user nova-compute[71323]:   </resource>
Apr 14 03:07:30 user nova-compute[71323]:   <sysinfo type='smbios'>
Apr 14 03:07:30 user nova-compute[71323]:     <system>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='version'>0.0.0</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='serial'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='uuid'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='family'>Virtual Machine</entry>
Apr 14 03:07:30 user nova-compute[71323]:     </system>
Apr 14 03:07:30 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:30 user nova-compute[71323]:   <os>
Apr 14 03:07:30 user nova-compute[71323]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 03:07:30 user nova-compute[71323]:     <boot dev='hd'/>
Apr 14 03:07:30 user nova-compute[71323]:     <smbios mode='sysinfo'/>
Apr 14 03:07:30 user nova-compute[71323]:   </os>
Apr 14 03:07:30 user nova-compute[71323]:   <features>
Apr 14 03:07:30 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:30 user nova-compute[71323]:     <apic/>
Apr 14 03:07:30 user nova-compute[71323]:     <vmcoreinfo state='on'/>
Apr 14 03:07:30 user nova-compute[71323]:   </features>
Apr 14 03:07:30 user nova-compute[71323]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 03:07:30 user nova-compute[71323]:     <model fallback='forbid'>Nehalem</model>
Apr 14 03:07:30 user nova-compute[71323]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='vme'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='x2apic'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='hypervisor'/>
Apr 14 03:07:30 user nova-compute[71323]:   </cpu>
Apr 14 03:07:30 user nova-compute[71323]:   <clock offset='utc'>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='hpet' present='no'/>
Apr 14 03:07:30 user nova-compute[71323]:   </clock>
Apr 14 03:07:30 user nova-compute[71323]:   <on_poweroff>destroy</on_poweroff>
Apr 14 03:07:30 user nova-compute[71323]:   <on_reboot>restart</on_reboot>
Apr 14 03:07:30 user nova-compute[71323]:   <on_crash>destroy</on_crash>
Apr 14 03:07:30 user nova-compute[71323]:   <devices>
Apr 14 03:07:30 user nova-compute[71323]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 03:07:30 user nova-compute[71323]:     <disk type='file' device='disk'>
Apr 14 03:07:30 user nova-compute[71323]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 03:07:30 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk' index='2'/>
Apr 14 03:07:30 user nova-compute[71323]:       <backingStore type='file' index='3'>
Apr 14 03:07:30 user nova-compute[71323]:         <format type='raw'/>
Apr 14 03:07:30 user nova-compute[71323]:         <source file='/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d'/>
Apr 14 03:07:30 user nova-compute[71323]:         <backingStore/>
Apr 14 03:07:30 user nova-compute[71323]:       </backingStore>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='vda' bus='virtio'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='virtio-disk0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </disk>
Apr 14 03:07:30 user nova-compute[71323]:     <disk type='file' device='cdrom'>
Apr 14 03:07:30 user nova-compute[71323]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 03:07:30 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config' index='1'/>
Apr 14 03:07:30 user nova-compute[71323]:       <backingStore/>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='hda' bus='ide'/>
Apr 14 03:07:30 user nova-compute[71323]:       <readonly/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='ide0-0-0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </disk>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='usb' index='0' model='none'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='usb'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='pci.0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='ide' index='0'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='ide'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <interface type='ethernet'>
Apr 14 03:07:30 user nova-compute[71323]:       <mac address='fa:16:3e:e7:c3:de'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='tapfb9a2be7-8d'/>
Apr 14 03:07:30 user nova-compute[71323]:       <model type='virtio'/>
Apr 14 03:07:30 user nova-compute[71323]:       <mtu size='1442'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='net0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </interface>
Apr 14 03:07:30 user nova-compute[71323]:     <interface type='ethernet'>
Apr 14 03:07:30 user nova-compute[71323]:       <mac address='fa:16:3e:fc:c1:87'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='tapbf026195-61'/>
Apr 14 03:07:30 user nova-compute[71323]:       <model type='virtio'/>
Apr 14 03:07:30 user nova-compute[71323]:       <mtu size='1442'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='net1'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </interface>
Apr 14 03:07:30 user nova-compute[71323]:     <serial type='pty'>
Apr 14 03:07:30 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:30 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target type='isa-serial' port='0'>
Apr 14 03:07:30 user nova-compute[71323]:         <model name='isa-serial'/>
Apr 14 03:07:30 user nova-compute[71323]:       </target>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </serial>
Apr 14 03:07:30 user nova-compute[71323]:     <console type='pty' tty='/dev/pts/4'>
Apr 14 03:07:30 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:30 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target type='serial' port='0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </console>
Apr 14 03:07:30 user nova-compute[71323]:     <input type='mouse' bus='ps2'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='input0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </input>
Apr 14 03:07:30 user nova-compute[71323]:     <input type='keyboard' bus='ps2'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='input1'/>
Apr 14 03:07:30 user nova-compute[71323]:     </input>
Apr 14 03:07:30 user nova-compute[71323]:     <graphics type='vnc' port='5904' autoport='yes' listen='0.0.0.0'>
Apr 14 03:07:30 user nova-compute[71323]:       <listen type='address' address='0.0.0.0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </graphics>
Apr 14 03:07:30 user nova-compute[71323]:     <audio id='1' type='none'/>
Apr 14 03:07:30 user nova-compute[71323]:     <video>
Apr 14 03:07:30 user nova-compute[71323]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='video0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </video>
Apr 14 03:07:30 user nova-compute[71323]:     <memballoon model='virtio'>
Apr 14 03:07:30 user nova-compute[71323]:       <stats period='10'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='balloon0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:30 user nova-compute[71323]:     <rng model='virtio'>
Apr 14 03:07:30 user nova-compute[71323]:       <backend model='random'>/dev/urandom</backend>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='rng0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </rng>
Apr 14 03:07:30 user nova-compute[71323]:   </devices>
Apr 14 03:07:30 user nova-compute[71323]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 03:07:30 user nova-compute[71323]:     <label>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</label>
Apr 14 03:07:30 user nova-compute[71323]:     <imagelabel>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</imagelabel>
Apr 14 03:07:30 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:30 user nova-compute[71323]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 03:07:30 user nova-compute[71323]:     <label>+64055:+108</label>
Apr 14 03:07:30 user nova-compute[71323]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 03:07:30 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:30 user nova-compute[71323]: </domain>
Apr 14 03:07:30 user nova-compute[71323]:  {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 03:07:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully detached device tapbf026195-61 from instance 3e73add1-26be-47dd-adc5-506ab517544a from the persistent domain config.
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] (1/8): Attempting to detach device tapbf026195-61 with device alias net1 from instance 3e73add1-26be-47dd-adc5-506ab517544a from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] detach device xml: <interface type="ethernet">
Apr 14 03:07:30 user nova-compute[71323]:   <mac address="fa:16:3e:fc:c1:87"/>
Apr 14 03:07:30 user nova-compute[71323]:   <model type="virtio"/>
Apr 14 03:07:30 user nova-compute[71323]:   <mtu size="1442"/>
Apr 14 03:07:30 user nova-compute[71323]:   <target dev="tapbf026195-61"/>
Apr 14 03:07:30 user nova-compute[71323]: </interface>
Apr 14 03:07:30 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441650.9059052, 3e73add1-26be-47dd-adc5-506ab517544a => net1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Start waiting for the detach event from libvirt for device tapbf026195-61 with device alias net1 for instance 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface>not found in domain: <domain type='kvm' id='24'>
Apr 14 03:07:30 user nova-compute[71323]:   <name>instance-00000013</name>
Apr 14 03:07:30 user nova-compute[71323]:   <uuid>3e73add1-26be-47dd-adc5-506ab517544a</uuid>
Apr 14 03:07:30 user nova-compute[71323]:   <metadata>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:30 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:19</nova:creationTime>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:30 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:30 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:30 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:port uuid="bf026195-619f-4db1-a57a-1b3c0c83eb50">
Apr 14 03:07:30 user nova-compute[71323]:       <nova:ip type="fixed" address="10.10.10.137" ipVersion="4"/>
Apr 14 03:07:30 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:30 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:30 user nova-compute[71323]:   </metadata>
Apr 14 03:07:30 user nova-compute[71323]:   <memory unit='KiB'>131072</memory>
Apr 14 03:07:30 user nova-compute[71323]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 03:07:30 user nova-compute[71323]:   <vcpu placement='static'>1</vcpu>
Apr 14 03:07:30 user nova-compute[71323]:   <resource>
Apr 14 03:07:30 user nova-compute[71323]:     <partition>/machine</partition>
Apr 14 03:07:30 user nova-compute[71323]:   </resource>
Apr 14 03:07:30 user nova-compute[71323]:   <sysinfo type='smbios'>
Apr 14 03:07:30 user nova-compute[71323]:     <system>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='version'>0.0.0</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='serial'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='uuid'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:30 user nova-compute[71323]:       <entry name='family'>Virtual Machine</entry>
Apr 14 03:07:30 user nova-compute[71323]:     </system>
Apr 14 03:07:30 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:30 user nova-compute[71323]:   <os>
Apr 14 03:07:30 user nova-compute[71323]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 03:07:30 user nova-compute[71323]:     <boot dev='hd'/>
Apr 14 03:07:30 user nova-compute[71323]:     <smbios mode='sysinfo'/>
Apr 14 03:07:30 user nova-compute[71323]:   </os>
Apr 14 03:07:30 user nova-compute[71323]:   <features>
Apr 14 03:07:30 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:30 user nova-compute[71323]:     <apic/>
Apr 14 03:07:30 user nova-compute[71323]:     <vmcoreinfo state='on'/>
Apr 14 03:07:30 user nova-compute[71323]:   </features>
Apr 14 03:07:30 user nova-compute[71323]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 03:07:30 user nova-compute[71323]:     <model fallback='forbid'>Nehalem</model>
Apr 14 03:07:30 user nova-compute[71323]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='vme'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='x2apic'/>
Apr 14 03:07:30 user nova-compute[71323]:     <feature policy='require' name='hypervisor'/>
Apr 14 03:07:30 user nova-compute[71323]:   </cpu>
Apr 14 03:07:30 user nova-compute[71323]:   <clock offset='utc'>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 03:07:30 user nova-compute[71323]:     <timer name='hpet' present='no'/>
Apr 14 03:07:30 user nova-compute[71323]:   </clock>
Apr 14 03:07:30 user nova-compute[71323]:   <on_poweroff>destroy</on_poweroff>
Apr 14 03:07:30 user nova-compute[71323]:   <on_reboot>restart</on_reboot>
Apr 14 03:07:30 user nova-compute[71323]:   <on_crash>destroy</on_crash>
Apr 14 03:07:30 user nova-compute[71323]:   <devices>
Apr 14 03:07:30 user nova-compute[71323]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 03:07:30 user nova-compute[71323]:     <disk type='file' device='disk'>
Apr 14 03:07:30 user nova-compute[71323]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 03:07:30 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk' index='2'/>
Apr 14 03:07:30 user nova-compute[71323]:       <backingStore type='file' index='3'>
Apr 14 03:07:30 user nova-compute[71323]:         <format type='raw'/>
Apr 14 03:07:30 user nova-compute[71323]:         <source file='/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d'/>
Apr 14 03:07:30 user nova-compute[71323]:         <backingStore/>
Apr 14 03:07:30 user nova-compute[71323]:       </backingStore>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='vda' bus='virtio'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='virtio-disk0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </disk>
Apr 14 03:07:30 user nova-compute[71323]:     <disk type='file' device='cdrom'>
Apr 14 03:07:30 user nova-compute[71323]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 03:07:30 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config' index='1'/>
Apr 14 03:07:30 user nova-compute[71323]:       <backingStore/>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='hda' bus='ide'/>
Apr 14 03:07:30 user nova-compute[71323]:       <readonly/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='ide0-0-0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </disk>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='usb' index='0' model='none'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='usb'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='pci.0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <controller type='ide' index='0'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='ide'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 03:07:30 user nova-compute[71323]:     </controller>
Apr 14 03:07:30 user nova-compute[71323]:     <interface type='ethernet'>
Apr 14 03:07:30 user nova-compute[71323]:       <mac address='fa:16:3e:e7:c3:de'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target dev='tapfb9a2be7-8d'/>
Apr 14 03:07:30 user nova-compute[71323]:       <model type='virtio'/>
Apr 14 03:07:30 user nova-compute[71323]:       <mtu size='1442'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='net0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </interface>
Apr 14 03:07:30 user nova-compute[71323]:     <serial type='pty'>
Apr 14 03:07:30 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:30 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target type='isa-serial' port='0'>
Apr 14 03:07:30 user nova-compute[71323]:         <model name='isa-serial'/>
Apr 14 03:07:30 user nova-compute[71323]:       </target>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </serial>
Apr 14 03:07:30 user nova-compute[71323]:     <console type='pty' tty='/dev/pts/4'>
Apr 14 03:07:30 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:30 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:30 user nova-compute[71323]:       <target type='serial' port='0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </console>
Apr 14 03:07:30 user nova-compute[71323]:     <input type='mouse' bus='ps2'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='input0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </input>
Apr 14 03:07:30 user nova-compute[71323]:     <input type='keyboard' bus='ps2'>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='input1'/>
Apr 14 03:07:30 user nova-compute[71323]:     </input>
Apr 14 03:07:30 user nova-compute[71323]:     <graphics type='vnc' port='5904' autoport='yes' listen='0.0.0.0'>
Apr 14 03:07:30 user nova-compute[71323]:       <listen type='address' address='0.0.0.0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </graphics>
Apr 14 03:07:30 user nova-compute[71323]:     <audio id='1' type='none'/>
Apr 14 03:07:30 user nova-compute[71323]:     <video>
Apr 14 03:07:30 user nova-compute[71323]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='video0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </video>
Apr 14 03:07:30 user nova-compute[71323]:     <memballoon model='virtio'>
Apr 14 03:07:30 user nova-compute[71323]:       <stats period='10'/>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='balloon0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:30 user nova-compute[71323]:     <rng model='virtio'>
Apr 14 03:07:30 user nova-compute[71323]:       <backend model='random'>/dev/urandom</backend>
Apr 14 03:07:30 user nova-compute[71323]:       <alias name='rng0'/>
Apr 14 03:07:30 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 03:07:30 user nova-compute[71323]:     </rng>
Apr 14 03:07:30 user nova-compute[71323]:   </devices>
Apr 14 03:07:30 user nova-compute[71323]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 03:07:30 user nova-compute[71323]:     <label>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</label>
Apr 14 03:07:30 user nova-compute[71323]:     <imagelabel>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</imagelabel>
Apr 14 03:07:30 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:30 user nova-compute[71323]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 03:07:30 user nova-compute[71323]:     <label>+64055:+108</label>
Apr 14 03:07:30 user nova-compute[71323]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 03:07:30 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:30 user nova-compute[71323]: </domain>
Apr 14 03:07:30 user nova-compute[71323]:  {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 03:07:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully detached device tapbf026195-61 from instance 3e73add1-26be-47dd-adc5-506ab517544a from the live domain config.
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG os_vif [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbf026195-61, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:07:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:30 user nova-compute[71323]: INFO os_vif [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61')
Apr 14 03:07:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:30 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:30</nova:creationTime>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:30 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:30 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:30 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:30 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:30 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:30 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:30 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:30 user nova-compute[71323]:  {{(pid=71323) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-unplugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-unplugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:31 user nova-compute[71323]: WARNING nova.compute.manager [req-3025b323-dcbe-46fa-a1fe-c78bd0c59824 req-8f0cec6d-f417-4851-b3be-c3a737ae351d service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-unplugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 for instance with vm_state active and task_state None.
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-deleted-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:31 user nova-compute[71323]: INFO nova.compute.manager [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Neutron deleted interface bf026195-619f-4db1-a57a-1b3c0c83eb50; detaching it from the instance and deleting it from the info cache
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquired lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.objects.instance [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Lazy-loading 'system_metadata' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.objects.instance [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Lazy-loading 'flavor' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface>not found in domain: <domain type='kvm' id='24'>
Apr 14 03:07:31 user nova-compute[71323]:   <name>instance-00000013</name>
Apr 14 03:07:31 user nova-compute[71323]:   <uuid>3e73add1-26be-47dd-adc5-506ab517544a</uuid>
Apr 14 03:07:31 user nova-compute[71323]:   <metadata>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:31 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:30</nova:creationTime>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:31 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:31 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:31 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:31 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:31 user nova-compute[71323]:   </metadata>
Apr 14 03:07:31 user nova-compute[71323]:   <memory unit='KiB'>131072</memory>
Apr 14 03:07:31 user nova-compute[71323]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 03:07:31 user nova-compute[71323]:   <vcpu placement='static'>1</vcpu>
Apr 14 03:07:31 user nova-compute[71323]:   <resource>
Apr 14 03:07:31 user nova-compute[71323]:     <partition>/machine</partition>
Apr 14 03:07:31 user nova-compute[71323]:   </resource>
Apr 14 03:07:31 user nova-compute[71323]:   <sysinfo type='smbios'>
Apr 14 03:07:31 user nova-compute[71323]:     <system>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='version'>0.0.0</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='serial'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='uuid'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='family'>Virtual Machine</entry>
Apr 14 03:07:31 user nova-compute[71323]:     </system>
Apr 14 03:07:31 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:31 user nova-compute[71323]:   <os>
Apr 14 03:07:31 user nova-compute[71323]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 03:07:31 user nova-compute[71323]:     <boot dev='hd'/>
Apr 14 03:07:31 user nova-compute[71323]:     <smbios mode='sysinfo'/>
Apr 14 03:07:31 user nova-compute[71323]:   </os>
Apr 14 03:07:31 user nova-compute[71323]:   <features>
Apr 14 03:07:31 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:31 user nova-compute[71323]:     <apic/>
Apr 14 03:07:31 user nova-compute[71323]:     <vmcoreinfo state='on'/>
Apr 14 03:07:31 user nova-compute[71323]:   </features>
Apr 14 03:07:31 user nova-compute[71323]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 03:07:31 user nova-compute[71323]:     <model fallback='forbid'>Nehalem</model>
Apr 14 03:07:31 user nova-compute[71323]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='vme'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='x2apic'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='hypervisor'/>
Apr 14 03:07:31 user nova-compute[71323]:   </cpu>
Apr 14 03:07:31 user nova-compute[71323]:   <clock offset='utc'>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='hpet' present='no'/>
Apr 14 03:07:31 user nova-compute[71323]:   </clock>
Apr 14 03:07:31 user nova-compute[71323]:   <on_poweroff>destroy</on_poweroff>
Apr 14 03:07:31 user nova-compute[71323]:   <on_reboot>restart</on_reboot>
Apr 14 03:07:31 user nova-compute[71323]:   <on_crash>destroy</on_crash>
Apr 14 03:07:31 user nova-compute[71323]:   <devices>
Apr 14 03:07:31 user nova-compute[71323]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 03:07:31 user nova-compute[71323]:     <disk type='file' device='disk'>
Apr 14 03:07:31 user nova-compute[71323]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 03:07:31 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk' index='2'/>
Apr 14 03:07:31 user nova-compute[71323]:       <backingStore type='file' index='3'>
Apr 14 03:07:31 user nova-compute[71323]:         <format type='raw'/>
Apr 14 03:07:31 user nova-compute[71323]:         <source file='/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d'/>
Apr 14 03:07:31 user nova-compute[71323]:         <backingStore/>
Apr 14 03:07:31 user nova-compute[71323]:       </backingStore>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='vda' bus='virtio'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='virtio-disk0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </disk>
Apr 14 03:07:31 user nova-compute[71323]:     <disk type='file' device='cdrom'>
Apr 14 03:07:31 user nova-compute[71323]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 03:07:31 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config' index='1'/>
Apr 14 03:07:31 user nova-compute[71323]:       <backingStore/>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='hda' bus='ide'/>
Apr 14 03:07:31 user nova-compute[71323]:       <readonly/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='ide0-0-0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </disk>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='usb' index='0' model='none'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='usb'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='pci.0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='ide' index='0'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='ide'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <interface type='ethernet'>
Apr 14 03:07:31 user nova-compute[71323]:       <mac address='fa:16:3e:e7:c3:de'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='tapfb9a2be7-8d'/>
Apr 14 03:07:31 user nova-compute[71323]:       <model type='virtio'/>
Apr 14 03:07:31 user nova-compute[71323]:       <mtu size='1442'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='net0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </interface>
Apr 14 03:07:31 user nova-compute[71323]:     <serial type='pty'>
Apr 14 03:07:31 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:31 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target type='isa-serial' port='0'>
Apr 14 03:07:31 user nova-compute[71323]:         <model name='isa-serial'/>
Apr 14 03:07:31 user nova-compute[71323]:       </target>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </serial>
Apr 14 03:07:31 user nova-compute[71323]:     <console type='pty' tty='/dev/pts/4'>
Apr 14 03:07:31 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:31 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target type='serial' port='0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </console>
Apr 14 03:07:31 user nova-compute[71323]:     <input type='mouse' bus='ps2'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='input0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </input>
Apr 14 03:07:31 user nova-compute[71323]:     <input type='keyboard' bus='ps2'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='input1'/>
Apr 14 03:07:31 user nova-compute[71323]:     </input>
Apr 14 03:07:31 user nova-compute[71323]:     <graphics type='vnc' port='5904' autoport='yes' listen='0.0.0.0'>
Apr 14 03:07:31 user nova-compute[71323]:       <listen type='address' address='0.0.0.0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </graphics>
Apr 14 03:07:31 user nova-compute[71323]:     <audio id='1' type='none'/>
Apr 14 03:07:31 user nova-compute[71323]:     <video>
Apr 14 03:07:31 user nova-compute[71323]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='video0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </video>
Apr 14 03:07:31 user nova-compute[71323]:     <memballoon model='virtio'>
Apr 14 03:07:31 user nova-compute[71323]:       <stats period='10'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='balloon0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:31 user nova-compute[71323]:     <rng model='virtio'>
Apr 14 03:07:31 user nova-compute[71323]:       <backend model='random'>/dev/urandom</backend>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='rng0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </rng>
Apr 14 03:07:31 user nova-compute[71323]:   </devices>
Apr 14 03:07:31 user nova-compute[71323]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 03:07:31 user nova-compute[71323]:     <label>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</label>
Apr 14 03:07:31 user nova-compute[71323]:     <imagelabel>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</imagelabel>
Apr 14 03:07:31 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:31 user nova-compute[71323]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 03:07:31 user nova-compute[71323]:     <label>+64055:+108</label>
Apr 14 03:07:31 user nova-compute[71323]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 03:07:31 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:31 user nova-compute[71323]: </domain>
Apr 14 03:07:31 user nova-compute[71323]:  {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface> {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:fc:c1:87"/><model type="virtio"/><mtu size="1442"/><target dev="tapbf026195-61"/></interface>not found in domain: <domain type='kvm' id='24'>
Apr 14 03:07:31 user nova-compute[71323]:   <name>instance-00000013</name>
Apr 14 03:07:31 user nova-compute[71323]:   <uuid>3e73add1-26be-47dd-adc5-506ab517544a</uuid>
Apr 14 03:07:31 user nova-compute[71323]:   <metadata>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:31 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:30</nova:creationTime>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:31 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:31 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:31 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:31 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:31 user nova-compute[71323]:   </metadata>
Apr 14 03:07:31 user nova-compute[71323]:   <memory unit='KiB'>131072</memory>
Apr 14 03:07:31 user nova-compute[71323]:   <currentMemory unit='KiB'>131072</currentMemory>
Apr 14 03:07:31 user nova-compute[71323]:   <vcpu placement='static'>1</vcpu>
Apr 14 03:07:31 user nova-compute[71323]:   <resource>
Apr 14 03:07:31 user nova-compute[71323]:     <partition>/machine</partition>
Apr 14 03:07:31 user nova-compute[71323]:   </resource>
Apr 14 03:07:31 user nova-compute[71323]:   <sysinfo type='smbios'>
Apr 14 03:07:31 user nova-compute[71323]:     <system>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='product'>OpenStack Nova</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='version'>0.0.0</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='serial'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='uuid'>3e73add1-26be-47dd-adc5-506ab517544a</entry>
Apr 14 03:07:31 user nova-compute[71323]:       <entry name='family'>Virtual Machine</entry>
Apr 14 03:07:31 user nova-compute[71323]:     </system>
Apr 14 03:07:31 user nova-compute[71323]:   </sysinfo>
Apr 14 03:07:31 user nova-compute[71323]:   <os>
Apr 14 03:07:31 user nova-compute[71323]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Apr 14 03:07:31 user nova-compute[71323]:     <boot dev='hd'/>
Apr 14 03:07:31 user nova-compute[71323]:     <smbios mode='sysinfo'/>
Apr 14 03:07:31 user nova-compute[71323]:   </os>
Apr 14 03:07:31 user nova-compute[71323]:   <features>
Apr 14 03:07:31 user nova-compute[71323]:     <acpi/>
Apr 14 03:07:31 user nova-compute[71323]:     <apic/>
Apr 14 03:07:31 user nova-compute[71323]:     <vmcoreinfo state='on'/>
Apr 14 03:07:31 user nova-compute[71323]:   </features>
Apr 14 03:07:31 user nova-compute[71323]:   <cpu mode='custom' match='exact' check='full'>
Apr 14 03:07:31 user nova-compute[71323]:     <model fallback='forbid'>Nehalem</model>
Apr 14 03:07:31 user nova-compute[71323]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='vme'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='x2apic'/>
Apr 14 03:07:31 user nova-compute[71323]:     <feature policy='require' name='hypervisor'/>
Apr 14 03:07:31 user nova-compute[71323]:   </cpu>
Apr 14 03:07:31 user nova-compute[71323]:   <clock offset='utc'>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='pit' tickpolicy='delay'/>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='rtc' tickpolicy='catchup'/>
Apr 14 03:07:31 user nova-compute[71323]:     <timer name='hpet' present='no'/>
Apr 14 03:07:31 user nova-compute[71323]:   </clock>
Apr 14 03:07:31 user nova-compute[71323]:   <on_poweroff>destroy</on_poweroff>
Apr 14 03:07:31 user nova-compute[71323]:   <on_reboot>restart</on_reboot>
Apr 14 03:07:31 user nova-compute[71323]:   <on_crash>destroy</on_crash>
Apr 14 03:07:31 user nova-compute[71323]:   <devices>
Apr 14 03:07:31 user nova-compute[71323]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Apr 14 03:07:31 user nova-compute[71323]:     <disk type='file' device='disk'>
Apr 14 03:07:31 user nova-compute[71323]:       <driver name='qemu' type='qcow2' cache='none'/>
Apr 14 03:07:31 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk' index='2'/>
Apr 14 03:07:31 user nova-compute[71323]:       <backingStore type='file' index='3'>
Apr 14 03:07:31 user nova-compute[71323]:         <format type='raw'/>
Apr 14 03:07:31 user nova-compute[71323]:         <source file='/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d'/>
Apr 14 03:07:31 user nova-compute[71323]:         <backingStore/>
Apr 14 03:07:31 user nova-compute[71323]:       </backingStore>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='vda' bus='virtio'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='virtio-disk0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </disk>
Apr 14 03:07:31 user nova-compute[71323]:     <disk type='file' device='cdrom'>
Apr 14 03:07:31 user nova-compute[71323]:       <driver name='qemu' type='raw' cache='none'/>
Apr 14 03:07:31 user nova-compute[71323]:       <source file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk.config' index='1'/>
Apr 14 03:07:31 user nova-compute[71323]:       <backingStore/>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='hda' bus='ide'/>
Apr 14 03:07:31 user nova-compute[71323]:       <readonly/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='ide0-0-0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </disk>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='usb' index='0' model='none'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='usb'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='pci' index='0' model='pci-root'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='pci.0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <controller type='ide' index='0'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='ide'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Apr 14 03:07:31 user nova-compute[71323]:     </controller>
Apr 14 03:07:31 user nova-compute[71323]:     <interface type='ethernet'>
Apr 14 03:07:31 user nova-compute[71323]:       <mac address='fa:16:3e:e7:c3:de'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target dev='tapfb9a2be7-8d'/>
Apr 14 03:07:31 user nova-compute[71323]:       <model type='virtio'/>
Apr 14 03:07:31 user nova-compute[71323]:       <mtu size='1442'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='net0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </interface>
Apr 14 03:07:31 user nova-compute[71323]:     <serial type='pty'>
Apr 14 03:07:31 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:31 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target type='isa-serial' port='0'>
Apr 14 03:07:31 user nova-compute[71323]:         <model name='isa-serial'/>
Apr 14 03:07:31 user nova-compute[71323]:       </target>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </serial>
Apr 14 03:07:31 user nova-compute[71323]:     <console type='pty' tty='/dev/pts/4'>
Apr 14 03:07:31 user nova-compute[71323]:       <source path='/dev/pts/4'/>
Apr 14 03:07:31 user nova-compute[71323]:       <log file='/opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/console.log' append='off'/>
Apr 14 03:07:31 user nova-compute[71323]:       <target type='serial' port='0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='serial0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </console>
Apr 14 03:07:31 user nova-compute[71323]:     <input type='mouse' bus='ps2'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='input0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </input>
Apr 14 03:07:31 user nova-compute[71323]:     <input type='keyboard' bus='ps2'>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='input1'/>
Apr 14 03:07:31 user nova-compute[71323]:     </input>
Apr 14 03:07:31 user nova-compute[71323]:     <graphics type='vnc' port='5904' autoport='yes' listen='0.0.0.0'>
Apr 14 03:07:31 user nova-compute[71323]:       <listen type='address' address='0.0.0.0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </graphics>
Apr 14 03:07:31 user nova-compute[71323]:     <audio id='1' type='none'/>
Apr 14 03:07:31 user nova-compute[71323]:     <video>
Apr 14 03:07:31 user nova-compute[71323]:       <model type='virtio' heads='1' primary='yes'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='video0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </video>
Apr 14 03:07:31 user nova-compute[71323]:     <memballoon model='virtio'>
Apr 14 03:07:31 user nova-compute[71323]:       <stats period='10'/>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='balloon0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </memballoon>
Apr 14 03:07:31 user nova-compute[71323]:     <rng model='virtio'>
Apr 14 03:07:31 user nova-compute[71323]:       <backend model='random'>/dev/urandom</backend>
Apr 14 03:07:31 user nova-compute[71323]:       <alias name='rng0'/>
Apr 14 03:07:31 user nova-compute[71323]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Apr 14 03:07:31 user nova-compute[71323]:     </rng>
Apr 14 03:07:31 user nova-compute[71323]:   </devices>
Apr 14 03:07:31 user nova-compute[71323]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Apr 14 03:07:31 user nova-compute[71323]:     <label>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</label>
Apr 14 03:07:31 user nova-compute[71323]:     <imagelabel>libvirt-3e73add1-26be-47dd-adc5-506ab517544a</imagelabel>
Apr 14 03:07:31 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:31 user nova-compute[71323]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Apr 14 03:07:31 user nova-compute[71323]:     <label>+64055:+108</label>
Apr 14 03:07:31 user nova-compute[71323]:     <imagelabel>+64055:+108</imagelabel>
Apr 14 03:07:31 user nova-compute[71323]:   </seclabel>
Apr 14 03:07:31 user nova-compute[71323]: </domain>
Apr 14 03:07:31 user nova-compute[71323]:  {{(pid=71323) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Apr 14 03:07:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Detaching interface fa:16:3e:fc:c1:87 failed because the device is no longer found on the guest.: nova.exception.DeviceNotFound: Device 'tapbf026195-61' not found.
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Converting VIF {"id": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "address": "fa:16:3e:fc:c1:87", "network": {"id": "891ae671-db4d-49dd-92c7-efc22ae86a34", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-268565756", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.137", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbf026195-61", "ovs_interfaceid": "bf026195-619f-4db1-a57a-1b3c0c83eb50", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG os_vif [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbf026195-61, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:07:31 user nova-compute[71323]: INFO os_vif [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fc:c1:87,bridge_name='br-int',has_traffic_filtering=True,id=bf026195-619f-4db1-a57a-1b3c0c83eb50,network=Network(891ae671-db4d-49dd-92c7-efc22ae86a34),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbf026195-61')
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [req-7e5e5ecf-bd8d-45cd-b834-7d50f8efe7c3 req-55f961a7-8672-45b0-a44d-0313a0d2eae4 service nova] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:07:31 user nova-compute[71323]:   <nova:package version="0.0.0"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:name>tempest-device-tagging-server-1844029430</nova:name>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:creationTime>2023-04-14 03:07:31</nova:creationTime>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:flavor name="m1.nano">
Apr 14 03:07:31 user nova-compute[71323]:     <nova:memory>128</nova:memory>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:disk>1</nova:disk>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:swap>0</nova:swap>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:vcpus>1</nova:vcpus>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:flavor>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:user uuid="17999fdd4eb449d49d74b4de20f6614a">tempest-TaggedAttachmentsTest-1835109148-project-member</nova:user>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:project uuid="6a991a4fd13b4a819b24479a341ba08c">tempest-TaggedAttachmentsTest-1835109148</nova:project>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:owner>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:07:31 user nova-compute[71323]:   <nova:ports>
Apr 14 03:07:31 user nova-compute[71323]:     <nova:port uuid="fb9a2be7-8d62-4528-a23c-bfdf995bd6dc">
Apr 14 03:07:31 user nova-compute[71323]:       <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:07:31 user nova-compute[71323]:     </nova:port>
Apr 14 03:07:31 user nova-compute[71323]:   </nova:ports>
Apr 14 03:07:31 user nova-compute[71323]: </nova:instance>
Apr 14 03:07:31 user nova-compute[71323]:  {{(pid=71323) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Apr 14 03:07:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [{"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:07:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Releasing lock "refresh_cache-3e73add1-26be-47dd-adc5-506ab517544a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:07:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-25803be4-0682-4a87-a90e-f51825bd728e tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "interface-3e73add1-26be-47dd-adc5-506ab517544a-bf026195-619f-4db1-a57a-1b3c0c83eb50" "released" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: held 1.521s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:07:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:07:33 user nova-compute[71323]: WARNING nova.compute.manager [req-51bb6a4d-6907-429f-8a5e-47fcfe30e8b6 req-3b37f326-8b18-4d78-9834-1852143235a2 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-bf026195-619f-4db1-a57a-1b3c0c83eb50 for instance with vm_state active and task_state None.
Apr 14 03:07:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:39 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441644.8930306, bbb6cd58-ce7a-43ae-876d-51a888193f60 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:07:39 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] VM Stopped (Lifecycle Event)
Apr 14 03:07:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-496dab72-2170-4f18-bbf0-822fb27700b5 None None] [instance: bbb6cd58-ce7a-43ae-876d-51a888193f60] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:07:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:54 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:07:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:07:58 user nova-compute[71323]: INFO nova.compute.claims [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Claim successful on node user
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.458s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:07:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:07:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:07:58 user nova-compute[71323]: INFO nova.virt.block_device [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Booting with blank volume at /dev/vda
Apr 14 03:07:58 user nova-compute[71323]: DEBUG nova.policy [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd666936eb0ee43a2ad9e17a12e6e273d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd2c5b2aafaf4ca7bf62e988ac726658', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Successfully created port: 3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:07:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Successfully updated port: 3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-changed-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Refreshing instance network info cache due to event network-changed-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance network_info: |[{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Refreshing network info cache for port 3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG nova.network.neutron [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updated VIF entry in instance network info cache for port 3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG nova.network.neutron [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8122c68f-534d-4417-9c75-d72fea68d883 req-97c37b05-d116-446d-a104-f215af0e5da1 service nova] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdc) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:03 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:03 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7952MB free_disk=26.536983489990234GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Applying migration context for instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 as it has an incoming, in-progress migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef. Migration status is finished {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 03:08:03 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating resource usage from migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 3e73add1-26be-47dd-adc5-506ab517544a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:08:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.479s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.utils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4967aae1-7204-4897-9cff-de4cfa51d9cd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d30a608-dc43-400b-9c8f-13de40789b0d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8835c832-fa49-4d82-bb2f-d3e50d9f0256]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2b7eb98-7b61-42c7-aaf5-df8f027c4823]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:04 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:08:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8cf913ab-5efe-4f07-808a-cb516c2a65a8]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG os_brick.utils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:04 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating existing volume attachment record: 54363199-122c-4acb-8e4a-9a1e390c992e {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:08:05 user nova-compute[71323]: INFO nova.compute.claims [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Claim successful on node user
Apr 14 03:08:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.413s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:08:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:08:06 user nova-compute[71323]: INFO nova.virt.block_device [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Booting with volume c6fbfee2-70fa-477a-a761-8d092061e691 at /dev/vda
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.policy [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.utils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f25b122f-62cb-4307-8613-018c47da786f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[090d3453-3ed7-48a8-962f-8e74a9842a8e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[43013721-0226-458e-9fc4-874f212627b0]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[50ae78f0-ba34-4b0b-9b67-d0ec8311386c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:06 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:08:06 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4211da32-4509-484e-8faf-82434706fad2]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG os_brick.utils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (155ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:06 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating existing volume attachment record: d90b3770-b8d3-4a03-8ecd-9841fb06a5f8 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:08:07 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Successfully created port: 3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Successfully updated port: 3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-changed-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Refreshing instance network info cache due to event network-changed-3ca37a66-26b1-4ab0-99fa-96237e59345f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] Acquiring lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating instance_info_cache with network_info: [{"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Instance network_info: |[{"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] Acquired lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.network.neutron [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Refreshing network info cache for port 3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:08:09 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Creating image(s)
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Ensure instance console log exists: /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start _get_guest_xml network_info=[{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '54363199-122c-4acb-8e4a-9a1e390c992e', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '2692a422-82e6-4f3b-b91e-6a2ea28e4d0a', 'attached_at': '', 'detached_at': '', 'volume_id': '0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'serial': '0af4cdb2-c179-4459-8ef7-43518caaaf7c'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.network.neutron [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updated VIF entry in instance network info cache for port 3ca37a66-26b1-4ab0-99fa-96237e59345f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.network.neutron [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating instance_info_cache with network_info: [{"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:09 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:09 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:09 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:09 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:09 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a728ab7c-8473-44ba-ade0-7aa7ae3efe26 req-325b9754-f95d-4f6d-a12f-4ce3cb00467c service nova] Releasing lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c87d772c-3a40-49c2-9746-f7cdb4477b9a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:08:09 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[584a1a7d-ee6c-4a7e-bb95-9a9089599e90]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c] added
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4066a7ad-d1e2-4a52-b64d-1b0196d84f5a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6fdc3c2-8d39-42d9-ab23-18f0ba90947f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kMFlEcQj {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kMFlEcQj" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e9490cd-9882-4f76-85e4-6974a13d164c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kMFlEcQj'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a099d787-d638-4a54-8767-e8ab440056b9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3900a2b-242b-450d-961b-1fd4d216aef1]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b40fbf3-1a5a-491b-8a62-cb41a7c41317]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260]
Apr 14 03:08:09 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcd6f6f4-d80b-4232-b89c-436268a0b7a4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[34536bd2-a127-4377-9bb5-9d4bb3cd243f]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:09 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.298s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] HCTL ('37', '-', '-', 0) found on session 26 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[03830999-fa09-4c23-82c3-c740fe03761b]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Searching for a device in session 26 and hctl ['37', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Searching for a device in session 26 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.330s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23966373662313666', 'path': '/dev/sdf'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23966373662313666', 'path': '/dev/sdf'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:07:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1770273062',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1770273062',id=22,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-reuivjtu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:07:59Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=2692a422-82e6-4f3b-b91e-6a2ea28e4d0a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.objects.instance [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:10 user nova-compute[71323]:   <uuid>2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</uuid>
Apr 14 03:08:10 user nova-compute[71323]:   <name>instance-00000016</name>
Apr 14 03:08:10 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:10 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:10 user nova-compute[71323]:   <metadata>
Apr 14 03:08:10 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:10 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1770273062</nova:name>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:09</nova:creationTime>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:10 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:10 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:08:10 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:08:10 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:10 user nova-compute[71323]:         <nova:port uuid="3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5">
Apr 14 03:08:10 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 03:08:10 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:10 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:10 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:10 user nova-compute[71323]:   </metadata>
Apr 14 03:08:10 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:10 user nova-compute[71323]:     <system>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="serial">2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</entry>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="uuid">2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</entry>
Apr 14 03:08:10 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:10 user nova-compute[71323]:     </system>
Apr 14 03:08:10 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:10 user nova-compute[71323]:   <os>
Apr 14 03:08:10 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:10 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:08:10 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:10 user nova-compute[71323]:   </os>
Apr 14 03:08:10 user nova-compute[71323]:   <features>
Apr 14 03:08:10 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:10 user nova-compute[71323]:     <apic/>
Apr 14 03:08:10 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:10 user nova-compute[71323]:   </features>
Apr 14 03:08:10 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:10 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:10 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:10 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:10 user nova-compute[71323]:   </clock>
Apr 14 03:08:10 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:10 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:10 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:10 user nova-compute[71323]:   </cpu>
Apr 14 03:08:10 user nova-compute[71323]:   <devices>
Apr 14 03:08:10 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:08:10 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:08:10 user nova-compute[71323]:       <source dev="/dev/sdf"/>
Apr 14 03:08:10 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:10 user nova-compute[71323]:       <serial>0af4cdb2-c179-4459-8ef7-43518caaaf7c</serial>
Apr 14 03:08:10 user nova-compute[71323]:     </disk>
Apr 14 03:08:10 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:10 user nova-compute[71323]:       <mac address="fa:16:3e:86:2f:f6"/>
Apr 14 03:08:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:10 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:10 user nova-compute[71323]:       <target dev="tap3b7ac3bb-62"/>
Apr 14 03:08:10 user nova-compute[71323]:     </interface>
Apr 14 03:08:10 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:10 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/console.log" append="off"/>
Apr 14 03:08:10 user nova-compute[71323]:     </serial>
Apr 14 03:08:10 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:10 user nova-compute[71323]:     <video>
Apr 14 03:08:10 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:10 user nova-compute[71323]:     </video>
Apr 14 03:08:10 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:10 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:10 user nova-compute[71323]:     </rng>
Apr 14 03:08:10 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:10 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:10 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:10 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:10 user nova-compute[71323]:   </devices>
Apr 14 03:08:10 user nova-compute[71323]: </domain>
Apr 14 03:08:10 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:07:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1770273062',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1770273062',id=22,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-reuivjtu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:07:59Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=2692a422-82e6-4f3b-b91e-6a2ea28e4d0a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG os_vif [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3b7ac3bb-62, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3b7ac3bb-62, col_values=(('external_ids', {'iface-id': '3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:86:2f:f6', 'vm-uuid': '2692a422-82e6-4f3b-b91e-6a2ea28e4d0a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:10 user nova-compute[71323]: INFO os_vif [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62')
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:86:2f:f6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:08:11 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Creating image(s)
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Ensure instance console log exists: /opt/stack/data/nova/instances/7a1e2563-0dc3-4cc9-aa61-7cb321f28622/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Start _get_guest_xml network_info=[{"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'd90b3770-b8d3-4a03-8ecd-9841fb06a5f8', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '7a1e2563-0dc3-4cc9-aa61-7cb321f28622', 'attached_at': '', 'detached_at': '', 'volume_id': 'c6fbfee2-70fa-477a-a761-8d092061e691', 'serial': 'c6fbfee2-70fa-477a-a761-8d092061e691'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:11 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:11 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:11 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d085f61a-df80-4318-9e51-5fd9ed1135e6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:08:11 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[68e1732d-d3fb-44b3-8f92-742a35989f9c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691] added
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f3124a1-98c6-4a12-bd9a-6a7c53870b17]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9896847-f7b8-45c0-9c55-3fdce1a93741]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tekamKoO {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tekamKoO" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6834923-73b1-4f89-a706-b63182920697]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tekamKoO'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1d795fb-a983-423e-9972-d30d3b123f7d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[392cee72-67dd-4d03-b45a-31bb8cc5a1bc]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[585eb385-11f5-4d56-8197-55cdefee9b10]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]
Apr 14 03:08:11 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[85d8126d-1d82-46ed-8540-24e365e6f2b7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b10b766-7593-4f15-8f65-ee53549b9dd5]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:11 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.307s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('38', '-', '-', 0) found on session 27 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed56bca2-3790-439b-bd08-bc56f933c46b]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 27 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:08:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:12 user nova-compute[71323]: WARNING nova.compute.manager [req-3f3a076c-35e3-4406-95e5-55d0f7a766b2 req-dcc6ed9f-4c4e-4758-85c7-8635259eac18 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state building and task_state spawning.
Apr 14 03:08:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.341s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1343ms) {'type': 'block', 'scsi_wwn': '23433326361643831', 'path': '/dev/sdg'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23433326361643831', 'path': '/dev/sdg'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1962356101',display_name='tempest-TestVolumeBootPattern-server-1962356101',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1962356101',id=23,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-gpx3ghi0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:06Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7a1e2563-0dc3-4cc9-aa61-7cb321f28622,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:12 user nova-compute[71323]:   <uuid>7a1e2563-0dc3-4cc9-aa61-7cb321f28622</uuid>
Apr 14 03:08:12 user nova-compute[71323]:   <name>instance-00000017</name>
Apr 14 03:08:12 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:12 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:12 user nova-compute[71323]:   <metadata>
Apr 14 03:08:12 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:12 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:12 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-server-1962356101</nova:name>
Apr 14 03:08:12 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:11</nova:creationTime>
Apr 14 03:08:12 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:12 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:12 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:12 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 03:08:12 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:12 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:12 user nova-compute[71323]:         <nova:port uuid="3ca37a66-26b1-4ab0-99fa-96237e59345f">
Apr 14 03:08:12 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:08:12 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:12 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:12 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:12 user nova-compute[71323]:   </metadata>
Apr 14 03:08:12 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:12 user nova-compute[71323]:     <system>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="serial">7a1e2563-0dc3-4cc9-aa61-7cb321f28622</entry>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="uuid">7a1e2563-0dc3-4cc9-aa61-7cb321f28622</entry>
Apr 14 03:08:12 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:12 user nova-compute[71323]:     </system>
Apr 14 03:08:12 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:12 user nova-compute[71323]:   <os>
Apr 14 03:08:12 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:12 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:08:12 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:12 user nova-compute[71323]:   </os>
Apr 14 03:08:12 user nova-compute[71323]:   <features>
Apr 14 03:08:12 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:12 user nova-compute[71323]:     <apic/>
Apr 14 03:08:12 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:12 user nova-compute[71323]:   </features>
Apr 14 03:08:12 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:12 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:12 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:12 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:12 user nova-compute[71323]:   </clock>
Apr 14 03:08:12 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:12 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:12 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:12 user nova-compute[71323]:   </cpu>
Apr 14 03:08:12 user nova-compute[71323]:   <devices>
Apr 14 03:08:12 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:08:12 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:08:12 user nova-compute[71323]:       <source dev="/dev/sdg"/>
Apr 14 03:08:12 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:12 user nova-compute[71323]:       <serial>c6fbfee2-70fa-477a-a761-8d092061e691</serial>
Apr 14 03:08:12 user nova-compute[71323]:     </disk>
Apr 14 03:08:12 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:12 user nova-compute[71323]:       <mac address="fa:16:3e:f1:dc:cc"/>
Apr 14 03:08:12 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:12 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:12 user nova-compute[71323]:       <target dev="tap3ca37a66-26"/>
Apr 14 03:08:12 user nova-compute[71323]:     </interface>
Apr 14 03:08:12 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:12 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/7a1e2563-0dc3-4cc9-aa61-7cb321f28622/console.log" append="off"/>
Apr 14 03:08:12 user nova-compute[71323]:     </serial>
Apr 14 03:08:12 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:12 user nova-compute[71323]:     <video>
Apr 14 03:08:12 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:12 user nova-compute[71323]:     </video>
Apr 14 03:08:12 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:12 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:12 user nova-compute[71323]:     </rng>
Apr 14 03:08:12 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:12 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:12 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:12 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:12 user nova-compute[71323]:   </devices>
Apr 14 03:08:12 user nova-compute[71323]: </domain>
Apr 14 03:08:12 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1962356101',display_name='tempest-TestVolumeBootPattern-server-1962356101',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1962356101',id=23,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-gpx3ghi0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:06Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7a1e2563-0dc3-4cc9-aa61-7cb321f28622,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG os_vif [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3ca37a66-26, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap3ca37a66-26, col_values=(('external_ids', {'iface-id': '3ca37a66-26b1-4ab0-99fa-96237e59345f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f1:dc:cc', 'vm-uuid': '7a1e2563-0dc3-4cc9-aa61-7cb321f28622'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:12 user nova-compute[71323]: INFO os_vif [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26')
Apr 14 03:08:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:f1:dc:cc, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441694.6645129, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Resumed (Lifecycle Event)
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance spawned successfully.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441694.670134, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Started (Lifecycle Event)
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] No waiting events found dispatching network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:14 user nova-compute[71323]: WARNING nova.compute.manager [req-e90152a2-409a-4596-8bf1-bffb19126e18 req-8c0cb064-c6da-48c8-9bb4-ab34fe738b55 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received unexpected event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f for instance with vm_state building and task_state spawning.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:14 user nova-compute[71323]: WARNING nova.compute.manager [req-8d679135-7470-450d-9887-acb8b6265667 req-52f36403-9a30-46a1-893a-6a7c87607ddd service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state building and task_state spawning.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Took 5.41 seconds to spawn the instance on the hypervisor.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:14 user nova-compute[71323]: INFO nova.compute.manager [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Took 16.77 seconds to build instance.
Apr 14 03:08:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-389ef989-72f9-4d4a-be54-d3c283d3f5aa tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.871s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Instance spawned successfully.
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441696.6663394, 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] VM Resumed (Lifecycle Event)
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441696.6664236, 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] VM Started (Lifecycle Event)
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Took 5.32 seconds to spawn the instance on the hypervisor.
Apr 14 03:08:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:16 user nova-compute[71323]: INFO nova.compute.manager [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Took 10.98 seconds to build instance.
Apr 14 03:08:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-185f7a07-56f5-4b21-a88c-e7b09596a2e6 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.080s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] No waiting events found dispatching network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:17 user nova-compute[71323]: WARNING nova.compute.manager [req-2c362945-03b3-447d-aca4-8d957b4bbf5e req-5372a4a2-376a-4627-8f3e-5518548f3922 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received unexpected event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f for instance with vm_state active and task_state None.
Apr 14 03:08:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:18 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:08:18 user nova-compute[71323]: INFO nova.compute.claims [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Claim successful on node user
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.452s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:08:19 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.policy [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd6d78d4991b5493e8893034e91ed407b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '130665008cf648cf87467ae2777a9170', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:08:19 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Creating image(s)
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "/opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "/opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "/opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk 1073741824" returned: 0 in 0.058s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.207s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Cannot resize image /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.objects.instance [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'migration_context' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Ensure instance console log exists: /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:20 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Successfully created port: cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Successfully updated port: cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquired lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-changed-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Refreshing instance network info cache due to event network-changed-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] Acquiring lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating instance_info_cache with network_info: [{"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Releasing lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Instance network_info: |[{"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] Acquired lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.neutron [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Refreshing network info cache for port cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Start _get_guest_xml network_info=[{"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:21 user nova-compute[71323]: INFO nova.compute.manager [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Terminating instance
Apr 14 03:08:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1987903443',display_name='tempest-AttachVolumeTestJSON-server-1987903443',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1987903443',id=24,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDFjzX0+40uO37CfqkOBSmbMWwyV5wsfc6L7vAuDWYejkbH4JLkWAIz1Wn0uLtB1OjrTaShOAhE1miwOi7ekeCgSgaeKQp/3FeYMl2Pn1H8x1Gn7ZQIusaMOU4fzl1tfrQ==',key_name='tempest-keypair-1327512481',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-quz7070b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=9c4bbe1e-d297-40e2-95d5-d8ab381c302d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.objects.instance [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'pci_devices' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:21 user nova-compute[71323]:   <uuid>9c4bbe1e-d297-40e2-95d5-d8ab381c302d</uuid>
Apr 14 03:08:21 user nova-compute[71323]:   <name>instance-00000018</name>
Apr 14 03:08:21 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:21 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:21 user nova-compute[71323]:   <metadata>
Apr 14 03:08:21 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:21 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeTestJSON-server-1987903443</nova:name>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:21</nova:creationTime>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:21 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:21 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:user uuid="d6d78d4991b5493e8893034e91ed407b">tempest-AttachVolumeTestJSON-975876676-project-member</nova:user>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:project uuid="130665008cf648cf87467ae2777a9170">tempest-AttachVolumeTestJSON-975876676</nova:project>
Apr 14 03:08:21 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:08:21 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:21 user nova-compute[71323]:         <nova:port uuid="cf6c0efd-f0d7-4a17-837f-eefb2b1a5731">
Apr 14 03:08:21 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 03:08:21 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:21 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:21 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:21 user nova-compute[71323]:   </metadata>
Apr 14 03:08:21 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:21 user nova-compute[71323]:     <system>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="serial">9c4bbe1e-d297-40e2-95d5-d8ab381c302d</entry>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="uuid">9c4bbe1e-d297-40e2-95d5-d8ab381c302d</entry>
Apr 14 03:08:21 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:21 user nova-compute[71323]:     </system>
Apr 14 03:08:21 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:21 user nova-compute[71323]:   <os>
Apr 14 03:08:21 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:21 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:08:21 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:21 user nova-compute[71323]:   </os>
Apr 14 03:08:21 user nova-compute[71323]:   <features>
Apr 14 03:08:21 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:21 user nova-compute[71323]:     <apic/>
Apr 14 03:08:21 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:21 user nova-compute[71323]:   </features>
Apr 14 03:08:21 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:21 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:21 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:21 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:21 user nova-compute[71323]:   </clock>
Apr 14 03:08:21 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:21 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:21 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:21 user nova-compute[71323]:   </cpu>
Apr 14 03:08:21 user nova-compute[71323]:   <devices>
Apr 14 03:08:21 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:08:21 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:08:21 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk"/>
Apr 14 03:08:21 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:21 user nova-compute[71323]:     </disk>
Apr 14 03:08:21 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:21 user nova-compute[71323]:       <mac address="fa:16:3e:5d:ff:d3"/>
Apr 14 03:08:21 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:21 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:21 user nova-compute[71323]:       <target dev="tapcf6c0efd-f0"/>
Apr 14 03:08:21 user nova-compute[71323]:     </interface>
Apr 14 03:08:21 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:21 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/console.log" append="off"/>
Apr 14 03:08:21 user nova-compute[71323]:     </serial>
Apr 14 03:08:21 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:21 user nova-compute[71323]:     <video>
Apr 14 03:08:21 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:21 user nova-compute[71323]:     </video>
Apr 14 03:08:21 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:21 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:21 user nova-compute[71323]:     </rng>
Apr 14 03:08:21 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:21 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:21 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:21 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:21 user nova-compute[71323]:   </devices>
Apr 14 03:08:21 user nova-compute[71323]: </domain>
Apr 14 03:08:21 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1987903443',display_name='tempest-AttachVolumeTestJSON-server-1987903443',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1987903443',id=24,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDFjzX0+40uO37CfqkOBSmbMWwyV5wsfc6L7vAuDWYejkbH4JLkWAIz1Wn0uLtB1OjrTaShOAhE1miwOi7ekeCgSgaeKQp/3FeYMl2Pn1H8x1Gn7ZQIusaMOU4fzl1tfrQ==',key_name='tempest-keypair-1327512481',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-quz7070b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=9c4bbe1e-d297-40e2-95d5-d8ab381c302d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG os_vif [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapcf6c0efd-f0, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapcf6c0efd-f0, col_values=(('external_ids', {'iface-id': 'cf6c0efd-f0d7-4a17-837f-eefb2b1a5731', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5d:ff:d3', 'vm-uuid': '9c4bbe1e-d297-40e2-95d5-d8ab381c302d'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: INFO os_vif [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0')
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No VIF found with MAC fa:16:3e:5d:ff:d3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updated VIF entry in instance network info cache for port cf6c0efd-f0d7-4a17-837f-eefb2b1a5731. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating instance_info_cache with network_info: [{"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5a607f1-9682-47d7-bf74-97e0d13e031a req-47c1d09d-d0f7-421d-b667-d724bc5f45b5 service nova] Releasing lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-unplugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-unplugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-92cc9b25-c295-461c-8f5e-788888e0d56f req-f433dfd0-dfa1-43b1-b63f-a104fe5effc9 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-unplugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'flavor' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:22 user nova-compute[71323]: INFO nova.compute.manager [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Attaching volume 05d362e8-1aaf-4bbb-8a9f-9f153920e00c to /dev/vdb
Apr 14 03:08:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Instance destroyed successfully.
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lazy-loading 'resources' on Instance uuid 3e73add1-26be-47dd-adc5-506ab517544a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:05:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1844029430',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-1844029430',id=19,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDiHBttIIVBNMgMjltJBU4uMIz2as3PoKWtxm6ulgCLbjLzG8l8vypUin+t8362oN6Cvbgl7ijzYK1kaa0BhTC5evtLLqD2wBmJI1qq6z0TNdkAKe43vfhblj5qmB0bwRg==',key_name='tempest-keypair-1544638969',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='6a991a4fd13b4a819b24479a341ba08c',ramdisk_id='',reservation_id='r-10c0p7tj',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-1835109148',owner_user_name='tempest-TaggedAttachmentsTest-1835109148-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:05:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='17999fdd4eb449d49d74b4de20f6614a',uuid=3e73add1-26be-47dd-adc5-506ab517544a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converting VIF {"id": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "address": "fa:16:3e:e7:c3:de", "network": {"id": "7f091b9e-2d65-4db6-bca2-3e1f90f68689", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-998592764-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.15", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "6a991a4fd13b4a819b24479a341ba08c", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfb9a2be7-8d", "ovs_interfaceid": "fb9a2be7-8d62-4528-a23c-bfdf995bd6dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_vif [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfb9a2be7-8d, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:22 user nova-compute[71323]: INFO os_vif [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e7:c3:de,bridge_name='br-int',has_traffic_filtering=True,id=fb9a2be7-8d62-4528-a23c-bfdf995bd6dc,network=Network(7f091b9e-2d65-4db6-bca2-3e1f90f68689),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfb9a2be7-8d')
Apr 14 03:08:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Deleting instance files /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a_del
Apr 14 03:08:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Deletion of /opt/stack/data/nova/instances/3e73add1-26be-47dd-adc5-506ab517544a_del complete
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.utils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f5d658a-1c50-493f-b6ee-a6791af52a1b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1e41ec6-b7be-44a7-afab-72a7a2ebdc20]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:22 user nova-compute[71323]: INFO nova.compute.manager [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Took 1.23 seconds to destroy the instance on the hypervisor.
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8c92a7a-dadd-4fef-b41f-6a3a2d73b330]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[83079116-f363-4ab9-8d63-d39e3a9eb13f]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:22 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:08:22 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c23fe75c-3fd4-4a04-a639-14dc0c2775b0]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG os_brick.utils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:22 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating existing volume attachment record: f6acf2ef-4b28-4065-bb47-80775d83e693 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:08:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing instance network info cache due to event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:24 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Took 1.56 seconds to deallocate network for instance.
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5fd7af6-81eb-4b01-9aa0-cf3f2c2da1ef req-5535f4a9-feb6-47e2-8643-23fd6fee6dfe service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-deleted-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updated VIF entry in instance network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.601s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Acquiring lock "3e73add1-26be-47dd-adc5-506ab517544a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "3e73add1-26be-47dd-adc5-506ab517544a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] No waiting events found dispatching network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:25 user nova-compute[71323]: WARNING nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Received unexpected event network-vif-plugged-fb9a2be7-8d62-4528-a23c-bfdf995bd6dc for instance with vm_state active and task_state deleting.
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] No waiting events found dispatching network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:25 user nova-compute[71323]: WARNING nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received unexpected event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 for instance with vm_state building and task_state spawning.
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] No waiting events found dispatching network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:25 user nova-compute[71323]: WARNING nova.compute.manager [req-f7dab195-16c7-4877-bd77-61121de67d43 req-5dd311b5-3ae2-4a48-9295-240d511cc135 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received unexpected event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 for instance with vm_state building and task_state spawning.
Apr 14 03:08:25 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Deleted allocations for instance 3e73add1-26be-47dd-adc5-506ab517544a
Apr 14 03:08:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7677c39-ff67-4617-b98e-39e9a163f57f tempest-TaggedAttachmentsTest-1835109148 tempest-TaggedAttachmentsTest-1835109148-project-member] Lock "3e73add1-26be-47dd-adc5-506ab517544a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.662s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441705.9519365, 9c4bbe1e-d297-40e2-95d5-d8ab381c302d => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:25 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] VM Resumed (Lifecycle Event)
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:08:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Instance spawned successfully.
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:26 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441705.9537542, 9c4bbe1e-d297-40e2-95d5-d8ab381c302d => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:26 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] VM Started (Lifecycle Event)
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:26 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:26 user nova-compute[71323]: INFO nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Took 6.54 seconds to spawn the instance on the hypervisor.
Apr 14 03:08:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:26 user nova-compute[71323]: INFO nova.compute.manager [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Took 7.36 seconds to build instance.
Apr 14 03:08:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-73f66d54-3d07-42c4-8d86-8ecf076c7829 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.469s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KcHRlyjP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:27 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:27 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:27 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:08:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260" returned: 21 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac327e55-b651-4fd8-a119-526638c61eb1]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:08:27 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[60e05495-ba8e-49bb-8d25-0cda63a84f4d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c] added
Apr 14 03:08:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba6cb05b-662a-417a-82f5-e902f12a2124]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28beebd5-eaa4-42e0-bc43-6e107d8b0598]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KcHRlyjP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v KcHRlyjP" returned: 0 in 0.055s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e115371b-ca4b-4a50-862f-ca990cc0b8a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KcHRlyjP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b19c90a5-e402-4571-ad0f-fb8b022bf08c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d2fbedd-87ff-4404-b2b6-f2b164015c02]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:27 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:27 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --login" returned: 0 in 0.091s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a1e982c-22a9-4d16-baee-93ede142f87c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260]
Apr 14 03:08:28 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[931ad726-1314-410a-b14b-76d42c7494b8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f5afcec-b766-4f08-b822-735bcdc476d7]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.376s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] HCTL ('39', '-', '-', 0) found on session 28 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9193fe96-7512-4485-8166-945c4512bd74]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Searching for a device in session 28 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KcHRlyjP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:08:29 user nova-compute[71323]: INFO nova.compute.claims [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Claim successful on node user
Apr 14 03:08:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.414s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] <== connect_volume: return (1416ms) {'type': 'block', 'scsi_wwn': '26531336563613839', 'path': '/dev/sdh'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26531336563613839', 'path': '/dev/sdh'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.objects.instance [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'flavor' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:08:29 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:08:29 user nova-compute[71323]:   <source dev="/dev/sdh"/>
Apr 14 03:08:29 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:08:29 user nova-compute[71323]:   <serial>05d362e8-1aaf-4bbb-8a9f-9f153920e00c</serial>
Apr 14 03:08:29 user nova-compute[71323]: </disk>
Apr 14 03:08:29 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.514s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:08:29 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:3f:e5:e9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.policy [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0128a733bd6a4120addba5eda1cc05fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4549f6080dcf4c739ac03ef643801fbc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:08:29 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Creating image(s)
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "/opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "/opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "/opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.013s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-86adadb3-34dd-4d9d-94ca-a6e0393f7464 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.189s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk 1073741824" returned: 0 in 0.053s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.229s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Checking if we can resize image /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Cannot resize image /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'migration_context' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Ensure instance console log exists: /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Successfully created port: 136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:08:30 user nova-compute[71323]: INFO nova.compute.manager [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Rescuing
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Successfully updated port: 136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquired lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-changed-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Refreshing instance network info cache due to event network-changed-136dada7-395e-410e-8eee-03e4d82dc01a. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] Acquiring lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:31 user nova-compute[71323]: WARNING nova.compute.manager [req-8a949331-cfae-45e8-9ed0-8b6908d16ab3 req-9a9aff7a-2808-43d0-96f0-02326c6cc804 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state active and task_state rescuing.
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updating instance_info_cache with network_info: [{"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Releasing lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Instance network_info: |[{"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] Acquired lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Refreshing network info cache for port 136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Start _get_guest_xml network_info=[{"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-2069905968',display_name='tempest-VolumesSnapshotTestJSON-instance-2069905968',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-2069905968',id=25,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-b6bvow3h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:30Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=e1493ecb-e6a8-48c5-abf3-d3cbc092736a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'pci_devices' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:31 user nova-compute[71323]:   <uuid>e1493ecb-e6a8-48c5-abf3-d3cbc092736a</uuid>
Apr 14 03:08:31 user nova-compute[71323]:   <name>instance-00000019</name>
Apr 14 03:08:31 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:31 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:31 user nova-compute[71323]:   <metadata>
Apr 14 03:08:31 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:31 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-2069905968</nova:name>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:31</nova:creationTime>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:31 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:31 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:user uuid="0128a733bd6a4120addba5eda1cc05fb">tempest-VolumesSnapshotTestJSON-423841644-project-member</nova:user>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:project uuid="4549f6080dcf4c739ac03ef643801fbc">tempest-VolumesSnapshotTestJSON-423841644</nova:project>
Apr 14 03:08:31 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:08:31 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:31 user nova-compute[71323]:         <nova:port uuid="136dada7-395e-410e-8eee-03e4d82dc01a">
Apr 14 03:08:31 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 03:08:31 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:31 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:31 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:31 user nova-compute[71323]:   </metadata>
Apr 14 03:08:31 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:31 user nova-compute[71323]:     <system>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="serial">e1493ecb-e6a8-48c5-abf3-d3cbc092736a</entry>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="uuid">e1493ecb-e6a8-48c5-abf3-d3cbc092736a</entry>
Apr 14 03:08:31 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:31 user nova-compute[71323]:     </system>
Apr 14 03:08:31 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:31 user nova-compute[71323]:   <os>
Apr 14 03:08:31 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:31 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:08:31 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:31 user nova-compute[71323]:   </os>
Apr 14 03:08:31 user nova-compute[71323]:   <features>
Apr 14 03:08:31 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:31 user nova-compute[71323]:     <apic/>
Apr 14 03:08:31 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:31 user nova-compute[71323]:   </features>
Apr 14 03:08:31 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:31 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:31 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:31 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:31 user nova-compute[71323]:   </clock>
Apr 14 03:08:31 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:31 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:31 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:31 user nova-compute[71323]:   </cpu>
Apr 14 03:08:31 user nova-compute[71323]:   <devices>
Apr 14 03:08:31 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:08:31 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:08:31 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk"/>
Apr 14 03:08:31 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:31 user nova-compute[71323]:     </disk>
Apr 14 03:08:31 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:31 user nova-compute[71323]:       <mac address="fa:16:3e:17:63:b9"/>
Apr 14 03:08:31 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:31 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:31 user nova-compute[71323]:       <target dev="tap136dada7-39"/>
Apr 14 03:08:31 user nova-compute[71323]:     </interface>
Apr 14 03:08:31 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:31 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/console.log" append="off"/>
Apr 14 03:08:31 user nova-compute[71323]:     </serial>
Apr 14 03:08:31 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:31 user nova-compute[71323]:     <video>
Apr 14 03:08:31 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:31 user nova-compute[71323]:     </video>
Apr 14 03:08:31 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:31 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:31 user nova-compute[71323]:     </rng>
Apr 14 03:08:31 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:31 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:31 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:31 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:31 user nova-compute[71323]:   </devices>
Apr 14 03:08:31 user nova-compute[71323]: </domain>
Apr 14 03:08:31 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-2069905968',display_name='tempest-VolumesSnapshotTestJSON-instance-2069905968',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-2069905968',id=25,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-b6bvow3h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:30Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=e1493ecb-e6a8-48c5-abf3-d3cbc092736a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG os_vif [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap136dada7-39, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap136dada7-39, col_values=(('external_ids', {'iface-id': '136dada7-395e-410e-8eee-03e4d82dc01a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:17:63:b9', 'vm-uuid': 'e1493ecb-e6a8-48c5-abf3-d3cbc092736a'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:31 user nova-compute[71323]: INFO os_vif [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39')
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No VIF found with MAC fa:16:3e:17:63:b9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.network.neutron [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updated VIF entry in instance network info cache for port 136dada7-395e-410e-8eee-03e4d82dc01a. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.network.neutron [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updating instance_info_cache with network_info: [{"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39fce5fd-e75b-4c9a-b537-571cf5816a29 req-b41b7109-aa2e-4231-813d-54f2084b1b72 service nova] Releasing lock "refresh_cache-e1493ecb-e6a8-48c5-abf3-d3cbc092736a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:32 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance destroyed successfully.
Apr 14 03:08:32 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Attempting rescue
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:08:32 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Creating image(s)
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'trusted_certs' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.176s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue" returned: 0 in 0.073s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.261s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'migration_context' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start _get_guest_xml network_info=[{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:3f:e5:e9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'numa_topology' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'vcpu_model' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:06:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-413600185',display_name='tempest-ServerRescueNegativeTestJSON-server-413600185',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-413600185',id=20,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNRt1vkGxga6d2lnbq6r7KuWDr3P9XxrdKyfybFxOJEVdZ9m8h/yMxKRYhp7IzJglBQvcA3Veeq6pJJUHOvucLytIWZV1lh6jcIrnJ6ba4rOplP1XVeraWzy0zv10BSYng==',key_name='tempest-keypair-158606901',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:06:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-jnxbele2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:06:38Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d5e8150f1502427fb3148f20354d212e',uuid=4e47adee-bd94-4655-9bbd-ed59c5478531,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:3f:e5:e9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "vif_mac": "fa:16:3e:3f:e5:e9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'pci_devices' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:33 user nova-compute[71323]:   <uuid>4e47adee-bd94-4655-9bbd-ed59c5478531</uuid>
Apr 14 03:08:33 user nova-compute[71323]:   <name>instance-00000014</name>
Apr 14 03:08:33 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:33 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:33 user nova-compute[71323]:   <metadata>
Apr 14 03:08:33 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:33 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-413600185</nova:name>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:33</nova:creationTime>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:33 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:33 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:user uuid="d5e8150f1502427fb3148f20354d212e">tempest-ServerRescueNegativeTestJSON-773851452-project-member</nova:user>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:project uuid="558bfd7e1d9049d396381de148b0c4a5">tempest-ServerRescueNegativeTestJSON-773851452</nova:project>
Apr 14 03:08:33 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:08:33 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:33 user nova-compute[71323]:         <nova:port uuid="4773c5d2-1520-4fc1-ae46-f83a883cfd1b">
Apr 14 03:08:33 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:08:33 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:33 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:33 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:33 user nova-compute[71323]:   </metadata>
Apr 14 03:08:33 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:33 user nova-compute[71323]:     <system>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="serial">4e47adee-bd94-4655-9bbd-ed59c5478531</entry>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="uuid">4e47adee-bd94-4655-9bbd-ed59c5478531</entry>
Apr 14 03:08:33 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:33 user nova-compute[71323]:     </system>
Apr 14 03:08:33 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:33 user nova-compute[71323]:   <os>
Apr 14 03:08:33 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:33 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:33 user nova-compute[71323]:   </os>
Apr 14 03:08:33 user nova-compute[71323]:   <features>
Apr 14 03:08:33 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:33 user nova-compute[71323]:     <apic/>
Apr 14 03:08:33 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:33 user nova-compute[71323]:   </features>
Apr 14 03:08:33 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:33 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:33 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:33 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:33 user nova-compute[71323]:   </clock>
Apr 14 03:08:33 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:33 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:33 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:33 user nova-compute[71323]:   </cpu>
Apr 14 03:08:33 user nova-compute[71323]:   <devices>
Apr 14 03:08:33 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:08:33 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:08:33 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue"/>
Apr 14 03:08:33 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:33 user nova-compute[71323]:     </disk>
Apr 14 03:08:33 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:08:33 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:08:33 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk"/>
Apr 14 03:08:33 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:08:33 user nova-compute[71323]:     </disk>
Apr 14 03:08:33 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:33 user nova-compute[71323]:       <mac address="fa:16:3e:3f:e5:e9"/>
Apr 14 03:08:33 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:33 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:33 user nova-compute[71323]:       <target dev="tap4773c5d2-15"/>
Apr 14 03:08:33 user nova-compute[71323]:     </interface>
Apr 14 03:08:33 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:33 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/console.log" append="off"/>
Apr 14 03:08:33 user nova-compute[71323]:     </serial>
Apr 14 03:08:33 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:33 user nova-compute[71323]:     <video>
Apr 14 03:08:33 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:33 user nova-compute[71323]:     </video>
Apr 14 03:08:33 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:33 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:33 user nova-compute[71323]:     </rng>
Apr 14 03:08:33 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:33 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:33 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:33 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:33 user nova-compute[71323]:   </devices>
Apr 14 03:08:33 user nova-compute[71323]: </domain>
Apr 14 03:08:33 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance destroyed successfully.
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] No VIF found with MAC fa:16:3e:3f:e5:e9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:33 user nova-compute[71323]: WARNING nova.compute.manager [req-af7da8e1-8491-400e-8aff-d23650b9f78c req-40c4dbdb-4666-4f26-ac4e-c97eda33b4fc service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state active and task_state rescuing.
Apr 14 03:08:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] No waiting events found dispatching network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:34 user nova-compute[71323]: WARNING nova.compute.manager [req-dfb8422e-3daa-44d9-a5db-92a2cc2fce42 req-1c5edb1d-c8f4-4a94-90dd-f3f4453c34a1 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received unexpected event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a for instance with vm_state building and task_state spawning.
Apr 14 03:08:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:36 user nova-compute[71323]: WARNING nova.compute.manager [req-14e0d7c9-7780-40c9-91e8-777cc010343d req-6a5cea7d-e752-4c35-a484-976cb2b384d9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state active and task_state rescuing.
Apr 14 03:08:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] No waiting events found dispatching network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:36 user nova-compute[71323]: WARNING nova.compute.manager [req-28170ef6-bf61-4ca8-a3cc-fb8f7f995ea0 req-1244bfb0-86e4-4975-bc29-33279907863d service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received unexpected event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a for instance with vm_state building and task_state spawning.
Apr 14 03:08:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441702.7309825, 3e73add1-26be-47dd-adc5-506ab517544a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:37 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] VM Stopped (Lifecycle Event)
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba9e1d65-44e4-4117-9f20-358bdb250e10 None None] [instance: 3e73add1-26be-47dd-adc5-506ab517544a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441717.9284177, e1493ecb-e6a8-48c5-abf3-d3cbc092736a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] VM Resumed (Lifecycle Event)
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fe428e47-805a-4867-99f4-3ff1cabb7e87 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Instance spawned successfully.
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:08:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441717.9295554, e1493ecb-e6a8-48c5-abf3-d3cbc092736a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] VM Started (Lifecycle Event)
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Took 8.24 seconds to spawn the instance on the hypervisor.
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:38 user nova-compute[71323]: WARNING nova.compute.manager [req-e37de553-3691-4fff-9ad9-abfe2399eec2 req-4b2e82c9-7e6c-42ea-9e27-b953f3fb449c service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state rescued and task_state None.
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 4e47adee-bd94-4655-9bbd-ed59c5478531 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441717.945256, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Resumed (Lifecycle Event)
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441717.945671, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Started (Lifecycle Event)
Apr 14 03:08:38 user nova-compute[71323]: INFO nova.compute.manager [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Took 9.09 seconds to build instance.
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8f9087d3-0644-4725-b707-42a5ca7463e5 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.387s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Port 04b5c1f1-8988-4a34-b97a-ad0cecd12b71 binding to destination host user is already ACTIVE {{(pid=71323) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Creating tmpfile /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/tmp1m2htx04 to verify with other compute node that the instance is on the same shared storage. {{(pid=71323) check_instance_shared_storage_local /opt/stack/nova/nova/virt/libvirt/driver.py:9595}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:40 user nova-compute[71323]: WARNING nova.compute.manager [req-73710fdc-97f7-48c1-a4f0-f7f678e2367c req-d9081cf2-d838-4137-b006-13ea5a1cb739 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state resized and task_state resize_reverting.
Apr 14 03:08:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance destroyed successfully.
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={rebuild='server'},migration_context=<?>,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap04b5c1f1-89, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:40 user nova-compute[71323]: INFO os_vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:40 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b573bde4-6266-4c3b-bb83-c4e5cf09c691]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:08:40 user nova-compute[71323]: No targets found.
Apr 14 03:08:40 user nova-compute[71323]: iSNS:
Apr 14 03:08:40 user nova-compute[71323]: No targets found.
Apr 14 03:08:40 user nova-compute[71323]: STATIC:
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:08:40 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:40 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:40 user nova-compute[71323]: FIRMWARE:
Apr 14 03:08:40 user nova-compute[71323]: No targets found.
Apr 14 03:08:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:08:40 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:08:40 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fdbabaa-e3a7-49d9-8dbc-bf8136e9c3a3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[46f17674-1f17-4f6f-b250-a1ad72c7e3af]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:40 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:40 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd63b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-changed-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing instance network info cache due to event network-changed-2e49423b-86d6-4b5a-b9cf-321ee37937c3. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] Acquiring lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] Acquired lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Refreshing network info cache for port 2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[302bd4fa-0f6d-4d8d-aea7-ba18ffbcfc88]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.036s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f38c04c-e820-4ac4-8e07-367516141e8e]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f8381cf-3a03-4b56-b186-7aa8d0b31b2b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.041s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c6d44d0-4a89-44f2-afa3-658814003f55]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a70ddde-478a-4652-b35d-df70854ebb62]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout" returned: 0 in 0.092s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e0d04da-1103-478b-a800-13823b5eb05c]: (4, ('Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogout of [sid: 21, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:08:41 user nova-compute[71323]: Logout of [sid: 21, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.036s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fad240d8-5d14-4d1f-bb99-f7f343c31eca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.410s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (411ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG nova.network.neutron [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updated VIF entry in instance network info cache for port 2e49423b-86d6-4b5a-b9cf-321ee37937c3. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG nova.network.neutron [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c5bfd495-3b47-4cf0-a888-66d880018ee4 req-646a2806-7f30-4342-a2ee-3688b3c9989f service nova] Releasing lock "refresh_cache-29366fd0-eee1-4eb2-921a-c46b9677cc93" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'migration_context' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:42 user nova-compute[71323]: WARNING nova.compute.manager [req-04d455f9-cbf7-4fa3-a446-f755f0466d0c req-f2c8a18d-7844-41e0-8f2e-2412a46c2580 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state resized and task_state resize_reverting.
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: held 0.518s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:42 user nova-compute[71323]: INFO nova.compute.manager [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Terminating instance
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:08:42 user nova-compute[71323]: INFO nova.compute.manager [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Swapping old allocation on dict_keys(['8eb2236f-d4b5-4afe-871b-37575055ded0']) held by migration 81aa1fef-be2b-4f5a-be3d-174aad3ebdef for instance
Apr 14 03:08:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Overwriting current allocation {'allocations': {'8eb2236f-d4b5-4afe-871b-37575055ded0': {'resources': {'VCPU': 1, 'MEMORY_MB': 192, 'DISK_GB': 1}, 'generation': 29}}, 'project_id': '4075885e26cb4adaad24d3ae6217fe35', 'user_id': '7dea23743ba945c4a9d539b8e277c653', 'consumer_generation': 1} on consumer 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) move_allocations /opt/stack/nova/nova/scheduler/client/report.py:2018}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-642a7bcc-f28e-4235-b46f-21381469c322 req-63bb5aeb-8905-4b2f-ab5d-269470a8935a service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:44 user nova-compute[71323]: WARNING nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-0a6ecbf1-8127-469a-92cc-5b4673e3db51 for instance with vm_state active and task_state deleting.
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-dd090320-de86-40c8-bafb-515fa9c628e3 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:44 user nova-compute[71323]: WARNING nova.compute.manager [req-bf5b0cd7-8770-4117-9d8b-314a27209ae5 req-a54c5959-e7d6-4912-98c5-6032045f3714 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dd090320-de86-40c8-bafb-515fa9c628e3 for instance with vm_state active and task_state deleting.
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.utils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Instance destroyed successfully.
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lazy-loading 'resources' on Instance uuid 29366fd0-eee1-4eb2-921a-c46b9677cc93 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1cdc675-631b-454c-b2c2-2fc34a3b46bc]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "address": "fa:16:3e:5a:7c:e0", "network": {"id": "1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1637270232-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2e49423b-86", "ovs_interfaceid": "2e49423b-86d6-4b5a-b9cf-321ee37937c3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2e49423b-86, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[641dcecb-fe37-44b8-af7e-52820b18fef8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[50eac006-2ab9-4d57-8d38-665dbc30a6c4]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2e9699d-f995-4b73-a651-510545386e45]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:44 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5a:7c:e0,bridge_name='br-int',has_traffic_filtering=True,id=2e49423b-86d6-4b5a-b9cf-321ee37937c3,network=Network(1fac4ba0-7ffa-4c1e-b03e-4f6b4da1acc0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2e49423b-86')
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:08:44 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:08:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[852d6b13-5ce2-48f7-858c-63e43a81616a]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_brick.utils [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (205ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdbeb1c70-26, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:10:a2:b1,bridge_name='br-int',has_traffic_filtering=True,id=dbeb1c70-267b-4fde-b1a5-0e3567a11489,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapdbeb1c70-26')
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0a6ecbf1-81, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3c:51:36,bridge_name='br-int',has_traffic_filtering=True,id=0a6ecbf1-8127-469a-92cc-5b4673e3db51,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0a6ecbf1-81')
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap98ba28ec-b7, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ae:42:b8,bridge_name='br-int',has_traffic_filtering=True,id=98ba28ec-b77c-4a23-b003-a4d29371cf95,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap98ba28ec-b7')
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdd090320-de, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:61:e3:f3,bridge_name='br-int',has_traffic_filtering=True,id=dd090320-de86-40c8-bafb-515fa9c628e3,network=Network(896608cc-3794-4116-8493-dc55728a1840),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdd090320-de')
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6691f178-71, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:b5:68,bridge_name='br-int',has_traffic_filtering=True,id=6691f178-7199-44c1-9dd7-049cf205185b,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6691f178-71')
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='True',created_at=2023-04-14T03:06:49Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-875179344',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-device-tagging-server-875179344',id=21,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHcISVbpMfujGhoh7rbfetW/zcM/03NdmV8Cb0fw6DH3n7dkVxNJiOWLbbCwDZHTif2tW1OoRH8+60pwgTl4zPLJddaDbYKFEZIg+5EIzppcSOENhjHsGzD18LuB1mTHTg==',key_name='tempest-keypair-125552723',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f1d531e7f7004ebaa529cb60de3849e7',ramdisk_id='',reservation_id='r-3hy2rr7y',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-641324897',owner_user_name='tempest-TaggedBootDevicesTest_v242-641324897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:07:17Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5a311491e2924c49b01fd42458ec2fc2',uuid=29366fd0-eee1-4eb2-921a-c46b9677cc93,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converting VIF {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap093349f3-a5, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:45 user nova-compute[71323]: INFO os_vif [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:63:33:1a,bridge_name='br-int',has_traffic_filtering=True,id=093349f3-a544-487d-9927-f75c63ac2e9c,network=Network(865279bb-ea51-408b-9323-2dfeddec5e82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap093349f3-a5')
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dqJMIbXg', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[30446d27-a493-4869-a435-be70a34db614]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]: iSNS:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]: STATIC:
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: FIRMWARE:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:08:45 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:08:45 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3f0aaeb-fcb4-48e7-b8e1-9c38b2d04ef6]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b18d5bc1-14a2-4940-812c-f7206e7a75b2]: (4, ('tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cd6ec20>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[930e2490-f8a7-4a59-8961-8f97fa01c511]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fb76d75-b194-4984-bf66-41b8ae4dd631]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3208f135-83fb-4e25-8b90-d2c7e4c98916]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[92dad023-115e-4c5e-99ab-16930f99f1c4]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.075s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7c373e4-e799-411e-8069-5d7493bc556a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --logout" returned: 0 in 0.074s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[52820572-f921-4538-8eb9-65346a78fc29]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260]
Apr 14 03:08:45 user nova-compute[71323]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7ac81e3-c104-4bb3-ada5-ecd6969601f0 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7812b7b-7397-444a-8251-0e50dfe5e9e0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.417s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== disconnect_volume: return (418ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'akyODBJC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbb564c5-2b89-426d-b256-8cf5efcd6fd7]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]: iSNS:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]: STATIC:
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:08:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:45 user nova-compute[71323]: FIRMWARE:
Apr 14 03:08:45 user nova-compute[71323]: No targets found.
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:08:45 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:08:45 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[08eb1097-8ddc-461e-be21-671863344936]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[745e7f5a-c321-4c8f-bae4-7675ec891ebe]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:45 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00625c60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e22cd11-e297-4b6c-a25e-8f50255f3256]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5896f3cb-9815-407a-8492-99fd109202d1]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[24cab39d-8951-4f1f-94aa-80cb9fd50a7b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.046s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc566aaa-a135-4010-8df4-ffce4989d3af]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[47992eb6-817b-41c4-a0ac-b5eb499d48b0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8981ce1a-2661-4960-ae13-7c99ec491e34]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260]
Apr 14 03:08:45 user nova-compute[71323]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e178f132-e302-4ee5-8ca6-45972e24754a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[da9cffa0-e507-47f2-a5c5-03a7f73531bb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.379s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== disconnect_volume: return (380ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'PhzIAFeq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[df872825-3157-4cfb-9121-678367d27266]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:08:46 user nova-compute[71323]: No targets found.
Apr 14 03:08:46 user nova-compute[71323]: iSNS:
Apr 14 03:08:46 user nova-compute[71323]: No targets found.
Apr 14 03:08:46 user nova-compute[71323]: STATIC:
Apr 14 03:08:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:08:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:08:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:08:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee
Apr 14 03:08:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:08:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:08:46 user nova-compute[71323]: FIRMWARE:
Apr 14 03:08:46 user nova-compute[71323]: No targets found.
Apr 14 03:08:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:08:46 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:08:46 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[01ea1dc7-553b-4490-9dab-ef6d23d7406e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[58a7dd05-d338-4dc2-b499-f5f8543a2063]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:46 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00625c60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-2e49423b-86d6-4b5a-b9cf-321ee37937c3 for instance with vm_state active and task_state deleting.
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-dbeb1c70-267b-4fde-b1a5-0e3567a11489 for instance with vm_state active and task_state deleting.
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-98ba28ec-b77c-4a23-b003-a4d29371cf95 for instance with vm_state active and task_state deleting.
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-6691f178-7199-44c1-9dd7-049cf205185b for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8735b309-29f2-419e-a179-00de00ad3ddb req-f7bacd56-88d2-40ca-a913-d56c9d8c8420 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-6691f178-7199-44c1-9dd7-049cf205185b for instance with vm_state active and task_state deleting.
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9591a09-8f58-4de4-98a6-d691de54300d]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[def86381-8192-4616-8836-4887805ba97b]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[661ef04d-944d-47fb-b01c-91c1260c8c83]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.146s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5333467-3c94-4eb2-b32d-0504767b5d37]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a651db27-54a8-46f5-a3cc-956a2da03aa9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0506e6e2-d1b6-4d15-a422-eff1d4660e8a]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260]
Apr 14 03:08:46 user nova-compute[71323]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f9476602-ab81-47c8-aa31-ecc8e3c672ee -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[481c156e-6335-4cbe-b54b-2289fdb6d384]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.470s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] <== disconnect_volume: return (471ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:08:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Deleting instance files /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93_del
Apr 14 03:08:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Deletion of /opt/stack/data/nova/instances/29366fd0-eee1-4eb2-921a-c46b9677cc93_del complete
Apr 14 03:08:46 user nova-compute[71323]: INFO nova.compute.manager [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Took 3.77 seconds to destroy the instance on the hypervisor.
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:08:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-unplugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-unplugged-093349f3-a544-487d-9927-f75c63ac2e9c for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Acquiring lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] No waiting events found dispatching network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:48 user nova-compute[71323]: WARNING nova.compute.manager [req-f0624beb-0174-4720-bd5e-5a3be9895ee3 req-501f86c2-4d58-4cb1-8dee-3d95ccf406a3 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received unexpected event network-vif-plugged-093349f3-a544-487d-9927-f75c63ac2e9c for instance with vm_state active and task_state deleting.
Apr 14 03:08:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Starting finish_revert_migration {{(pid=71323) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11792}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start _get_guest_xml network_info=[{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'dc1ebff7-3718-4344-8f48-15d70c443eac', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4', 'attached_at': '2023-04-14T03:08:50.000000', 'detached_at': '', 'volume_id': '44f3cbc9-fd81-4185-aa74-0896739af983', 'serial': '44f3cbc9-fd81-4185-aa74-0896739af983'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:08:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:08:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:50 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:08:50 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:08:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee78f5d3-973e-4895-9707-de21c8ed342f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:08:50 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-deleted-2e49423b-86d6-4b5a-b9cf-321ee37937c3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:50 user nova-compute[71323]: INFO nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Neutron deleted interface 2e49423b-86d6-4b5a-b9cf-321ee37937c3; detaching it from the instance and deleting it from the info cache
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "dd090320-de86-40c8-bafb-515fa9c628e3", "address": "fa:16:3e:61:e3:f3", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdd090320-de", "ovs_interfaceid": "dd090320-de86-40c8-bafb-515fa9c628e3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3e291f4-e0e4-4908-849e-3a37eed1692d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983] added
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f93a6dbf-8d18-4a26-83b4-0d69b68aae2d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Detach interface failed, port_id=2e49423b-86d6-4b5a-b9cf-321ee37937c3, reason: Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-deleted-dd090320-de86-40c8-bafb-515fa9c628e3 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:50 user nova-compute[71323]: INFO nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Neutron deleted interface dd090320-de86-40c8-bafb-515fa9c628e3; detaching it from the instance and deleting it from the info cache
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "6691f178-7199-44c1-9dd7-049cf205185b", "address": "fa:16:3e:48:b5:68", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6691f178-71", "ovs_interfaceid": "6691f178-7199-44c1-9dd7-049cf205185b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[003ba411-db55-4a82-9294-5c6fb60e445a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Took 3.76 seconds to deallocate network for instance.
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Detach interface failed, port_id=dd090320-de86-40c8-bafb-515fa9c628e3, reason: Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-deleted-6691f178-7199-44c1-9dd7-049cf205185b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:50 user nova-compute[71323]: INFO nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Neutron deleted interface 6691f178-7199-44c1-9dd7-049cf205185b; detaching it from the instance and deleting it from the info cache
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "address": "fa:16:3e:ae:42:b8", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap98ba28ec-b7", "ovs_interfaceid": "98ba28ec-b77c-4a23-b003-a4d29371cf95", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v rCsYcqle" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[25e438c8-3754-4683-8341-936714110172]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'rCsYcqle'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Detach interface failed, port_id=6691f178-7199-44c1-9dd7-049cf205185b, reason: Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-deleted-98ba28ec-b77c-4a23-b003-a4d29371cf95 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:50 user nova-compute[71323]: INFO nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Neutron deleted interface 98ba28ec-b77c-4a23-b003-a4d29371cf95; detaching it from the instance and deleting it from the info cache
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "093349f3-a544-487d-9927-f75c63ac2e9c", "address": "fa:16:3e:63:33:1a", "network": {"id": "865279bb-ea51-408b-9323-2dfeddec5e82", "bridge": "br-int", "label": "tempest-device-tagging-net2-1986203743", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap093349f3-a5", "ovs_interfaceid": "093349f3-a544-487d-9927-f75c63ac2e9c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d290b44-c830-4291-8d32-6a84cde9881e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e3ea74f-c250-4d8b-b4f7-623decc7660d req-d0242dcb-3fb5-48b2-bb96-893b4a9a334d service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Detach interface failed, port_id=98ba28ec-b77c-4a23-b003-a4d29371cf95, reason: Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[165b448a-e891-4422-b5e6-1c23c2f1d501]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --login" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[79807bbe-b4cf-4a82-a3a4-2c18d011393f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:08:50 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[191d326a-dd94-4c45-a76a-100193f7328d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[57f4f89b-3961-44a5-8416-d7f8e8608ead]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:08:50 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:08:50 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.313s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('33', '-', '-', 0) found on session 29 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[10ffdb74-f5d5-4f1d-b623-b9d59e8c1d4f]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 29 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:08:50 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.344s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1345ms) {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26562356334656261', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:08:51 user nova-compute[71323]:   <uuid>6bb082bc-0c74-4149-9bd0-ceb09d7764d4</uuid>
Apr 14 03:08:51 user nova-compute[71323]:   <name>instance-00000005</name>
Apr 14 03:08:51 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:08:51 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:08:51 user nova-compute[71323]:   <metadata>
Apr 14 03:08:51 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:08:51 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-1052167823</nova:name>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:08:50</nova:creationTime>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:08:51 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:08:51 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:owner>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:08:51 user nova-compute[71323]:       </nova:owner>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:08:51 user nova-compute[71323]:       <nova:ports>
Apr 14 03:08:51 user nova-compute[71323]:         <nova:port uuid="04b5c1f1-8988-4a34-b97a-ad0cecd12b71">
Apr 14 03:08:51 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:08:51 user nova-compute[71323]:         </nova:port>
Apr 14 03:08:51 user nova-compute[71323]:       </nova:ports>
Apr 14 03:08:51 user nova-compute[71323]:     </nova:instance>
Apr 14 03:08:51 user nova-compute[71323]:   </metadata>
Apr 14 03:08:51 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:08:51 user nova-compute[71323]:     <system>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="serial">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="uuid">6bb082bc-0c74-4149-9bd0-ceb09d7764d4</entry>
Apr 14 03:08:51 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:08:51 user nova-compute[71323]:     </system>
Apr 14 03:08:51 user nova-compute[71323]:   </sysinfo>
Apr 14 03:08:51 user nova-compute[71323]:   <os>
Apr 14 03:08:51 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:08:51 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:08:51 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:08:51 user nova-compute[71323]:   </os>
Apr 14 03:08:51 user nova-compute[71323]:   <features>
Apr 14 03:08:51 user nova-compute[71323]:     <acpi/>
Apr 14 03:08:51 user nova-compute[71323]:     <apic/>
Apr 14 03:08:51 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:08:51 user nova-compute[71323]:   </features>
Apr 14 03:08:51 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:08:51 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:08:51 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:08:51 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:08:51 user nova-compute[71323]:   </clock>
Apr 14 03:08:51 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:08:51 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:08:51 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:08:51 user nova-compute[71323]:   </cpu>
Apr 14 03:08:51 user nova-compute[71323]:   <devices>
Apr 14 03:08:51 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:08:51 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:08:51 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk"/>
Apr 14 03:08:51 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:08:51 user nova-compute[71323]:     </disk>
Apr 14 03:08:51 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:08:51 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:08:51 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:08:51 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:08:51 user nova-compute[71323]:       <serial>44f3cbc9-fd81-4185-aa74-0896739af983</serial>
Apr 14 03:08:51 user nova-compute[71323]:     </disk>
Apr 14 03:08:51 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:08:51 user nova-compute[71323]:       <mac address="fa:16:3e:af:bb:dd"/>
Apr 14 03:08:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:51 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:08:51 user nova-compute[71323]:       <target dev="tap04b5c1f1-89"/>
Apr 14 03:08:51 user nova-compute[71323]:     </interface>
Apr 14 03:08:51 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:08:51 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/console.log" append="off"/>
Apr 14 03:08:51 user nova-compute[71323]:     </serial>
Apr 14 03:08:51 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:08:51 user nova-compute[71323]:     <video>
Apr 14 03:08:51 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:08:51 user nova-compute[71323]:     </video>
Apr 14 03:08:51 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:08:51 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:08:51 user nova-compute[71323]:     </rng>
Apr 14 03:08:51 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:08:51 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:08:51 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:08:51 user nova-compute[71323]:     </memballoon>
Apr 14 03:08:51 user nova-compute[71323]:   </devices>
Apr 14 03:08:51 user nova-compute[71323]: </domain>
Apr 14 03:08:51 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:07:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG os_vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap04b5c1f1-89, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap04b5c1f1-89, col_values=(('external_ids', {'iface-id': '04b5c1f1-8988-4a34-b97a-ad0cecd12b71', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:af:bb:dd', 'vm-uuid': '6bb082bc-0c74-4149-9bd0-ceb09d7764d4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:08:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:51 user nova-compute[71323]: INFO os_vif [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:08:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-7feaed1e-1e7a-492b-bfbd-e170f73ed2a4 req-852cd712-dc53-4115-9185-445a46d626c1 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Received event network-vif-deleted-093349f3-a544-487d-9927-f75c63ac2e9c {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:52 user nova-compute[71323]: INFO nova.compute.manager [req-7feaed1e-1e7a-492b-bfbd-e170f73ed2a4 req-852cd712-dc53-4115-9185-445a46d626c1 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Neutron deleted interface 093349f3-a544-487d-9927-f75c63ac2e9c; detaching it from the instance and deleting it from the info cache
Apr 14 03:08:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-7feaed1e-1e7a-492b-bfbd-e170f73ed2a4 req-852cd712-dc53-4115-9185-445a46d626c1 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Updating instance_info_cache with network_info: [{"id": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "address": "fa:16:3e:10:a2:b1", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.189", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdbeb1c70-26", "ovs_interfaceid": "dbeb1c70-267b-4fde-b1a5-0e3567a11489", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "address": "fa:16:3e:3c:51:36", "network": {"id": "896608cc-3794-4116-8493-dc55728a1840", "bridge": "br-int", "label": "tempest-device-tagging-net1-1661893997", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.110", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "f1d531e7f7004ebaa529cb60de3849e7", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0a6ecbf1-81", "ovs_interfaceid": "0a6ecbf1-8127-469a-92cc-5b4673e3db51", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-7feaed1e-1e7a-492b-bfbd-e170f73ed2a4 req-852cd712-dc53-4115-9185-445a46d626c1 service nova] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Detach interface failed, port_id=093349f3-a544-487d-9927-f75c63ac2e9c, reason: Instance 29366fd0-eee1-4eb2-921a-c46b9677cc93 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:08:52 user nova-compute[71323]: INFO nova.compute.manager [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Took 2.40 seconds to detach 3 volumes for instance.
Apr 14 03:08:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:53 user nova-compute[71323]: WARNING nova.compute.manager [req-c73f11a0-2a14-41ab-a84f-597cdb102882 req-cc0c0834-6a9a-4d9f-b401-61907a51def0 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state resized and task_state resize_reverting.
Apr 14 03:08:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.410s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:53 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Deleted allocations for instance 29366fd0-eee1-4eb2-921a-c46b9677cc93
Apr 14 03:08:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d7233161-c817-44fc-9ade-798939881a93 tempest-TaggedBootDevicesTest_v242-641324897 tempest-TaggedBootDevicesTest_v242-641324897-project-member] Lock "29366fd0-eee1-4eb2-921a-c46b9677cc93" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 10.563s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441735.1518023, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Resumed (Lifecycle Event)
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance running successfully.
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] finish_revert_migration finished successfully. {{(pid=71323) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11836}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441735.153998, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Started (Lifecycle Event)
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: resized, current task_state: resize_reverting, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:08:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:08:55 user nova-compute[71323]: WARNING nova.compute.manager [req-c488783d-bdc6-4e06-aa1b-0ccb9291d037 req-0a6bf4c4-1ac1-4cd8-81aa-1db5d734c65d service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state resized and task_state resize_reverting.
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Apr 14 03:08:55 user nova-compute[71323]: INFO nova.compute.manager [None req-fd47067f-0779-4dc4-9147-3c95c7c98e59 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance to original state: 'active'
Apr 14 03:08:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:08:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:08:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:08:59 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441724.6949108, 29366fd0-eee1-4eb2-921a-c46b9677cc93 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:08:59 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] VM Stopped (Lifecycle Event)
Apr 14 03:08:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-351c0821-7e52-4a44-8414-64fd96499abc None None] [instance: 29366fd0-eee1-4eb2-921a-c46b9677cc93] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:09:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:09:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [{"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-ac032176-63bc-457c-ad53-5b4a854eec34" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:09:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdg (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:09:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:09:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:09:05 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7814MB free_disk=26.503875732421875GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.556s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:09:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:49 user nova-compute[71323]: INFO nova.compute.manager [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Rescuing
Apr 14 03:09:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:09:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:09:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:09:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:09:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:09:50 user nova-compute[71323]: WARNING nova.compute.manager [req-9f0c0553-8afb-4cc0-a787-4afdfbc0c15c req-9251e2b5-bf57-4336-a35a-213a7bbf779e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state rescuing.
Apr 14 03:09:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance destroyed successfully.
Apr 14 03:09:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Attempting a stable device rescue
Apr 14 03:09:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:09:50 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Creating image(s)
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.010s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'trusted_certs' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.part --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG nova.virt.images [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] 673d2a71-c448-4ba0-ab84-2f243d2c7cf3 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:09:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.part /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:09:52 user nova-compute[71323]: WARNING nova.compute.manager [req-c2ddc2eb-6425-49da-a4f2-62dbbe372880 req-bb386ad7-10d1-4a48-9fb6-bff159be61f4 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state rescuing.
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.part /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.converted" returned: 0 in 0.749s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755.converted --force-share --output=json" returned: 0 in 0.171s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.079s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755 --force-share --output=json" returned: 0 in 0.167s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755,backing_fmt=raw /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755,backing_fmt=raw /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue" returned: 0 in 0.050s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "e2a0b14946e92e9e76c532f3d0a86d6addaca755" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.225s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'migration_context' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start _get_guest_xml network_info=[{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:86:2f:f6"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '673d2a71-c448-4ba0-ab84-2f243d2c7cf3', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '54363199-122c-4acb-8e4a-9a1e390c992e', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, 'status': 'reserved', 'instance': '2692a422-82e6-4f3b-b91e-6a2ea28e4d0a', 'attached_at': '', 'detached_at': '', 'volume_id': '0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'serial': '0af4cdb2-c179-4459-8ef7-43518caaaf7c'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'numa_topology' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:53 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:09:53 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'vcpu_model' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:09:53 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:09:53 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:09:53 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1a956ca-9f32-4f98-ab2f-12fe35eb08f3]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = kMFlEcQj\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 03:09:53 user nova-compute[71323]: node.name = iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:09:53 user nova-compute[71323]: node.tpgt = -1
Apr 14 03:09:53 user nova-compute[71323]: node.startup = automatic
Apr 14 03:09:53 user nova-compute[71323]: node.leading_login = No
Apr 14 03:09:53 user nova-compute[71323]: iface.iscsi_ifacename = default
Apr 14 03:09:53 user nova-compute[71323]: iface.net_ifacename = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.ipaddress = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.prefix_len = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.hwaddress = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.transport_name = tcp
Apr 14 03:09:53 user nova-compute[71323]: iface.initiatorname = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.vlan_id = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.vlan_priority = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.vlan_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.iface_num = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.mtu = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.port = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.bootproto = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.subnet_mask = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.gateway = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_alt_client_id = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_dns = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_learn_iqn = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_vendor_id_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_vendor_id = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.dhcp_slp_da = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.fragmentation = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.gratuitous_arp = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.incoming_forwarding = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.tos_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.tos = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.ttl = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.delayed_ack = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.tcp_nagle = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.tcp_wsf_state = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.tcp_wsf = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.tcp_timer_scale = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.tcp_timestamp = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.redirect = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.def_task_mgmt_timeout = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.header_digest = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.data_digest = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.immediate_data = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.initial_r2t = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.data_seq_inorder = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.data_pdu_inorder = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.erl = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.max_receive_data_len = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.first_burst_len = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.max_outstanding_r2t = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.max_burst_len = 0
Apr 14 03:09:53 user nova-compute[71323]: iface.chap_auth = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.bidi_chap = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.strict_login_compliance = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.discovery_auth = <empty>
Apr 14 03:09:53 user nova-compute[71323]: iface.discovery_logout = <empty>
Apr 14 03:09:53 user nova-compute[71323]: node.discovery_address = <empty>
Apr 14 03:09:53 user nova-compute[71323]: node.discovery_port = 0
Apr 14 03:09:53 user nova-compute[71323]: node.discovery_type = static
Apr 14 03:09:53 user nova-compute[71323]: node.session.initial_cmdsn = 0
Apr 14 03:09:53 user nova-compute[71323]: node.session.initial_login_retry_max = 8
Apr 14 03:09:53 user nova-compute[71323]: node.session.xmit_thread_priority = -20
Apr 14 03:09:53 user nova-compute[71323]: node.session.cmds_max = 128
Apr 14 03:09:53 user nova-compute[71323]: node.session.queue_depth = 32
Apr 14 03:09:53 user nova-compute[71323]: node.session.nr_sessions = 1
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.authmethod = CHAP
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.username = kMFlEcQj
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.password = ***
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.username_in = <empty>
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.password_in = <empty>
Apr 14 03:09:53 user nova-compute[71323]: node.session.auth.chap_algs = MD5
Apr 14 03:09:53 user nova-compute[71323]: node.session.timeo.replacement_timeout = 120
Apr 14 03:09:53 user nova-compute[71323]: node.session.err_timeo.abort_timeout = 15
Apr 14 03:09:53 user nova-compute[71323]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 03:09:53 user nova-compute[71323]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 03:09:53 user nova-compute[71323]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.FastAbort = Yes
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.InitialR2T = No
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.ImmediateData = Yes
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.MaxConnections = 1
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 03:09:53 user nova-compute[71323]: node.session.iscsi.ERL = 0
Apr 14 03:09:53 user nova-compute[71323]: node.session.scan = manual
Apr 14 03:09:53 user nova-compute[71323]: node.session.reopen_max = 0
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].address = 172.16.0.220
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].port = 3260
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].startup = manual
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].tcp.window_size = 524288
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].tcp.type_of_service = 0
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].timeo.logout_timeout = 15
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].timeo.login_timeout = 15
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].timeo.auth_timeout = 45
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.DataDigest = None
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.IFMarker = No
Apr 14 03:09:53 user nova-compute[71323]: node.conn[0].iscsi.OFMarker = No
Apr 14 03:09:53 user nova-compute[71323]: # END RECORD
Apr 14 03:09:53 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0619634f-ef78-4824-9933-a7be7e56c789]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[33068b6e-40b1-4c9d-b607-6e0e514e662a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kMFlEcQj {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kMFlEcQj" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab93361a-cbd1-4690-a363-93036bc19d10]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kMFlEcQj'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[40746121-2f99-4e7a-89ab-d1cd32a8f282]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[291e4668-09e0-4fd7-8da2-affe22c7a330]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:09:53 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:09:53 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:09:53 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.148s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] HCTL ('37', '0', '0', 0) found on session 26 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Scanning host 37 c: 0, t: 0, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[088173e9-bb50-41ea-a7f7-a15d8485285c]: (4, ('0 0 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Searching for a device in session 26 and hctl ('37', '0', '0', 0) yield: sdf {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.174s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== connect_volume: return (175ms) {'type': 'block', 'scsi_wwn': '23966373662313666', 'path': '/dev/sdf'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23966373662313666', 'path': '/dev/sdf'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:07:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1770273062',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1770273062',id=22,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:14Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-reuivjtu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:08:15Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=2692a422-82e6-4f3b-b91e-6a2ea28e4d0a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:86:2f:f6"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:86:2f:f6"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:09:53 user nova-compute[71323]:   <uuid>2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</uuid>
Apr 14 03:09:53 user nova-compute[71323]:   <name>instance-00000016</name>
Apr 14 03:09:53 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:09:53 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:09:53 user nova-compute[71323]:   <metadata>
Apr 14 03:09:53 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:09:53 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1770273062</nova:name>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:09:53</nova:creationTime>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:09:53 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:09:53 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:owner>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:09:53 user nova-compute[71323]:       </nova:owner>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:09:53 user nova-compute[71323]:       <nova:ports>
Apr 14 03:09:53 user nova-compute[71323]:         <nova:port uuid="3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5">
Apr 14 03:09:53 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Apr 14 03:09:53 user nova-compute[71323]:         </nova:port>
Apr 14 03:09:53 user nova-compute[71323]:       </nova:ports>
Apr 14 03:09:53 user nova-compute[71323]:     </nova:instance>
Apr 14 03:09:53 user nova-compute[71323]:   </metadata>
Apr 14 03:09:53 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:09:53 user nova-compute[71323]:     <system>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="serial">2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</entry>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="uuid">2692a422-82e6-4f3b-b91e-6a2ea28e4d0a</entry>
Apr 14 03:09:53 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:09:53 user nova-compute[71323]:     </system>
Apr 14 03:09:53 user nova-compute[71323]:   </sysinfo>
Apr 14 03:09:53 user nova-compute[71323]:   <os>
Apr 14 03:09:53 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:09:53 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:09:53 user nova-compute[71323]:   </os>
Apr 14 03:09:53 user nova-compute[71323]:   <features>
Apr 14 03:09:53 user nova-compute[71323]:     <acpi/>
Apr 14 03:09:53 user nova-compute[71323]:     <apic/>
Apr 14 03:09:53 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:09:53 user nova-compute[71323]:   </features>
Apr 14 03:09:53 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:09:53 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:09:53 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:09:53 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:09:53 user nova-compute[71323]:   </clock>
Apr 14 03:09:53 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:09:53 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:09:53 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:09:53 user nova-compute[71323]:   </cpu>
Apr 14 03:09:53 user nova-compute[71323]:   <devices>
Apr 14 03:09:53 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:09:53 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:09:53 user nova-compute[71323]:       <source dev="/dev/sdf"/>
Apr 14 03:09:53 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:09:53 user nova-compute[71323]:       <serial>0af4cdb2-c179-4459-8ef7-43518caaaf7c</serial>
Apr 14 03:09:53 user nova-compute[71323]:     </disk>
Apr 14 03:09:53 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:09:53 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:09:53 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue"/>
Apr 14 03:09:53 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:09:53 user nova-compute[71323]:       <boot order="1"/>
Apr 14 03:09:53 user nova-compute[71323]:     </disk>
Apr 14 03:09:53 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:09:53 user nova-compute[71323]:       <mac address="fa:16:3e:86:2f:f6"/>
Apr 14 03:09:53 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:09:53 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:09:53 user nova-compute[71323]:       <target dev="tap3b7ac3bb-62"/>
Apr 14 03:09:53 user nova-compute[71323]:     </interface>
Apr 14 03:09:53 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:09:53 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/console.log" append="off"/>
Apr 14 03:09:53 user nova-compute[71323]:     </serial>
Apr 14 03:09:53 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:09:53 user nova-compute[71323]:     <video>
Apr 14 03:09:53 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:09:53 user nova-compute[71323]:     </video>
Apr 14 03:09:53 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:09:53 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:09:53 user nova-compute[71323]:     </rng>
Apr 14 03:09:53 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:09:53 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:09:53 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:09:53 user nova-compute[71323]:     </memballoon>
Apr 14 03:09:53 user nova-compute[71323]:   </devices>
Apr 14 03:09:53 user nova-compute[71323]: </domain>
Apr 14 03:09:53 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:09:53 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance destroyed successfully.
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:86:2f:f6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:09:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:09:55 user nova-compute[71323]: WARNING nova.compute.manager [req-9855bfe2-d30f-4a86-9e9a-75e5a7b0fbde req-3797d432-8ff2-4be7-a604-715fd32f23e0 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state rescuing.
Apr 14 03:09:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:56 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:09:57 user nova-compute[71323]: WARNING nova.compute.manager [req-2d70236e-3974-43d2-bd72-077faab4b635 req-29a10b12-369e-4c0f-ab36-df38db759709 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state rescuing.
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441797.5546608, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:09:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Resumed (Lifecycle Event)
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-93b2d27c-dbba-4c62-8765-9d6752800c9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:09:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441797.5574186, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:09:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Started (Lifecycle Event)
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:09:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:09:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 03:09:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:09:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:09:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:59 user nova-compute[71323]: INFO nova.compute.manager [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Terminating instance
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-changed-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Refreshing instance network info cache due to event network-changed-3ca37a66-26b1-4ab0-99fa-96237e59345f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] Acquiring lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] Acquired lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.network.neutron [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Refreshing network info cache for port 3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-unplugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] No waiting events found dispatching network-vif-unplugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b8151e6-8c89-4aa3-a782-1bfe7dbaea63 req-ebd7b35a-96a6-4c05-b22a-b5aa16a1929f service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-unplugged-3ca37a66-26b1-4ab0-99fa-96237e59345f for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:09:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Instance destroyed successfully.
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1962356101',display_name='tempest-TestVolumeBootPattern-server-1962356101',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1962356101',id=23,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:16Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-gpx3ghi0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:17Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7a1e2563-0dc3-4cc9-aa61-7cb321f28622,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG os_vif [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3ca37a66-26, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:09:59 user nova-compute[71323]: INFO os_vif [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:dc:cc,bridge_name='br-int',has_traffic_filtering=True,id=3ca37a66-26b1-4ab0-99fa-96237e59345f,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3ca37a66-26')
Apr 14 03:09:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:09:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:09:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:09:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[024f36e8-b237-44c1-b1eb-6891fabf5722]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:10:00 user nova-compute[71323]: No targets found.
Apr 14 03:10:00 user nova-compute[71323]: iSNS:
Apr 14 03:10:00 user nova-compute[71323]: No targets found.
Apr 14 03:10:00 user nova-compute[71323]: STATIC:
Apr 14 03:10:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:10:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:10:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:10:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:10:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:00 user nova-compute[71323]: FIRMWARE:
Apr 14 03:10:00 user nova-compute[71323]: No targets found.
Apr 14 03:10:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:10:00 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:10:00 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebd14a6a-7c4f-4b52-9cc7-c0aff4410581]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f2c240c-defb-4332-860e-342372ca5b94]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:00 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b119bb6d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691'): ({'sdg'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdg {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[123c4b9f-59e9-40b9-b921-5cba24608164]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6655c88-33c8-486f-a11d-3f755ceda58c]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdg {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f696fc17-dedf-48f1-ad06-811b305cf0c7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.050s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bc33816-57e6-4c15-9298-3de8101be5aa]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdg have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_4291d670', '/dev/disk/by-id/scsi-23433326361643831', '/dev/disk/by-id/scsi-1SCST_BIO4291d670-432cad81504b77c1') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c940ae3-c14b-4d97-85b4-49ff1c6caa05]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updated VIF entry in instance network info cache for port 3ca37a66-26b1-4ab0-99fa-96237e59345f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating instance_info_cache with network_info: [{"id": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "address": "fa:16:3e:f1:dc:cc", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.12", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3ca37a66-26", "ovs_interfaceid": "3ca37a66-26b1-4ab0-99fa-96237e59345f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce734ae2-c177-4511-a78d-c134ba0a51bb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ecd706f5-2ca4-4f71-962b-2c1539506410 req-32dc0373-bc36-4947-bcfc-af78ca398ed7 service nova] Releasing lock "refresh_cache-7a1e2563-0dc3-4cc9-aa61-7cb321f28622" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d07a6950-cb81-4f05-a47e-a35ede2cc72b]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]
Apr 14 03:10:00 user nova-compute[71323]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.039s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2788d57-d202-4b5f-a7d0-f31a16b5983a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.406s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (407ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:10:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Deleting instance files /opt/stack/data/nova/instances/7a1e2563-0dc3-4cc9-aa61-7cb321f28622_del
Apr 14 03:10:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Deletion of /opt/stack/data/nova/instances/7a1e2563-0dc3-4cc9-aa61-7cb321f28622_del complete
Apr 14 03:10:00 user nova-compute[71323]: INFO nova.compute.manager [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Took 1.36 seconds to destroy the instance on the hypervisor.
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] Acquiring lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] No waiting events found dispatching network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:01 user nova-compute[71323]: WARNING nova.compute.manager [req-84c8e245-7501-429e-ab40-8f5fa9f49091 req-6b24e2b7-6e93-44a8-8034-9887d6510bdc service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received unexpected event network-vif-plugged-3ca37a66-26b1-4ab0-99fa-96237e59345f for instance with vm_state active and task_state deleting.
Apr 14 03:10:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:01 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Took 1.32 seconds to deallocate network for instance.
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-461cf38d-58bc-4727-baf6-699b84856eb8 req-0919f736-9c1e-4821-ae3d-591c0e0a8dfa service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Received event network-vif-deleted-3ca37a66-26b1-4ab0-99fa-96237e59345f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:01 user nova-compute[71323]: INFO nova.compute.manager [req-461cf38d-58bc-4727-baf6-699b84856eb8 req-0919f736-9c1e-4821-ae3d-591c0e0a8dfa service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Neutron deleted interface 3ca37a66-26b1-4ab0-99fa-96237e59345f; detaching it from the instance and deleting it from the info cache
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.network.neutron [req-461cf38d-58bc-4727-baf6-699b84856eb8 req-0919f736-9c1e-4821-ae3d-591c0e0a8dfa service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-461cf38d-58bc-4727-baf6-699b84856eb8 req-0919f736-9c1e-4821-ae3d-591c0e0a8dfa service nova] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Detach interface failed, port_id=3ca37a66-26b1-4ab0-99fa-96237e59345f, reason: Instance 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:10:02 user nova-compute[71323]: INFO nova.compute.manager [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Took 0.93 seconds to detach 1 volumes for instance.
Apr 14 03:10:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:10:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.465s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:03 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance 7a1e2563-0dc3-4cc9-aa61-7cb321f28622
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-94888dc0-d782-440c-9087-d21054e29ac5 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7a1e2563-0dc3-4cc9-aa61-7cb321f28622" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.263s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [{"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-6bb082bc-0c74-4149-9bd0-ceb09d7764d4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk.rescue --force-share --output=json" returned: 0 in 0.163s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.155s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json" returned: 0 in 0.171s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:07 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:07 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7834MB free_disk=26.421363830566406GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1689}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.499s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.374s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:07 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:10:07 user nova-compute[71323]: INFO nova.compute.claims [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Claim successful on node user
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.518s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:10:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.policy [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3c683fe5636b49858857033797ff8e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82eb4fbf55f04294a3d97800142bdca9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:10:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Creating image(s)
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk 1073741824" returned: 0 in 0.061s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.206s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Successfully created port: 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Cannot resize image /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'migration_context' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Ensure instance console log exists: /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-changed-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Refreshing instance network info cache due to event network-changed-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] Acquiring lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] Acquired lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG nova.network.neutron [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Refreshing network info cache for port cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:10:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Successfully updated port: 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing instance network info cache due to event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updated VIF entry in instance network info cache for port cf6c0efd-f0d7-4a17-837f-eefb2b1a5731. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating instance_info_cache with network_info: [{"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f478156a-c95e-4c59-bcc8-e8e8544c9cf6 req-0fef86e4-4c29-4542-a989-80090e18a6ad service nova] Releasing lock "refresh_cache-9c4bbe1e-d297-40e2-95d5-d8ab381c302d" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance network_info: |[{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start _get_guest_xml network_info=[{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:10:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-196777223',display_name='tempest-ServerStableDeviceRescueTest-server-196777223',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-196777223',id=26,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7w1V2h51K5WeC0lNrY7+q4q2XqyPZ1TB9opotq+DNhc9r2qQzgCUPc3xsU2/ON8yD/igMXO0IAbOHg83C68rdtapd1/KHmCO4Nj4cZtEPuSMjanMb8ZhuWuQX+Z2NrHw==',key_name='tempest-keypair-1447416444',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='82eb4fbf55f04294a3d97800142bdca9',ramdisk_id='',reservation_id='r-pniwrgja',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1642713032',owner_user_name='tempest-ServerStableDeviceRescueTest-1642713032-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3c683fe5636b49858857033797ff8e68',uuid=8a67985b-1938-4b7d-b486-13f6811611f3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converting VIF {"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.objects.instance [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'pci_devices' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:10:11 user nova-compute[71323]:   <uuid>8a67985b-1938-4b7d-b486-13f6811611f3</uuid>
Apr 14 03:10:11 user nova-compute[71323]:   <name>instance-0000001a</name>
Apr 14 03:10:11 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:10:11 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:10:11 user nova-compute[71323]:   <metadata>
Apr 14 03:10:11 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:10:11 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-196777223</nova:name>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:10:11</nova:creationTime>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:10:11 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:10:11 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:owner>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:user uuid="3c683fe5636b49858857033797ff8e68">tempest-ServerStableDeviceRescueTest-1642713032-project-member</nova:user>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:project uuid="82eb4fbf55f04294a3d97800142bdca9">tempest-ServerStableDeviceRescueTest-1642713032</nova:project>
Apr 14 03:10:11 user nova-compute[71323]:       </nova:owner>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:10:11 user nova-compute[71323]:       <nova:ports>
Apr 14 03:10:11 user nova-compute[71323]:         <nova:port uuid="2d081ef4-9756-4571-b28c-0d4582c58f0a">
Apr 14 03:10:11 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:10:11 user nova-compute[71323]:         </nova:port>
Apr 14 03:10:11 user nova-compute[71323]:       </nova:ports>
Apr 14 03:10:11 user nova-compute[71323]:     </nova:instance>
Apr 14 03:10:11 user nova-compute[71323]:   </metadata>
Apr 14 03:10:11 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:10:11 user nova-compute[71323]:     <system>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="serial">8a67985b-1938-4b7d-b486-13f6811611f3</entry>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="uuid">8a67985b-1938-4b7d-b486-13f6811611f3</entry>
Apr 14 03:10:11 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:10:11 user nova-compute[71323]:     </system>
Apr 14 03:10:11 user nova-compute[71323]:   </sysinfo>
Apr 14 03:10:11 user nova-compute[71323]:   <os>
Apr 14 03:10:11 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:10:11 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:10:11 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:10:11 user nova-compute[71323]:   </os>
Apr 14 03:10:11 user nova-compute[71323]:   <features>
Apr 14 03:10:11 user nova-compute[71323]:     <acpi/>
Apr 14 03:10:11 user nova-compute[71323]:     <apic/>
Apr 14 03:10:11 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:10:11 user nova-compute[71323]:   </features>
Apr 14 03:10:11 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:10:11 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:10:11 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:10:11 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:10:11 user nova-compute[71323]:   </clock>
Apr 14 03:10:11 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:10:11 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:10:11 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:10:11 user nova-compute[71323]:   </cpu>
Apr 14 03:10:11 user nova-compute[71323]:   <devices>
Apr 14 03:10:11 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:10:11 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:10:11 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk"/>
Apr 14 03:10:11 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:10:11 user nova-compute[71323]:     </disk>
Apr 14 03:10:11 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:10:11 user nova-compute[71323]:       <mac address="fa:16:3e:af:d1:f9"/>
Apr 14 03:10:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:11 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:10:11 user nova-compute[71323]:       <target dev="tap2d081ef4-97"/>
Apr 14 03:10:11 user nova-compute[71323]:     </interface>
Apr 14 03:10:11 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:10:11 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/console.log" append="off"/>
Apr 14 03:10:11 user nova-compute[71323]:     </serial>
Apr 14 03:10:11 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:10:11 user nova-compute[71323]:     <video>
Apr 14 03:10:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:11 user nova-compute[71323]:     </video>
Apr 14 03:10:11 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:10:11 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:10:11 user nova-compute[71323]:     </rng>
Apr 14 03:10:11 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:10:11 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:10:11 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:10:11 user nova-compute[71323]:     </memballoon>
Apr 14 03:10:11 user nova-compute[71323]:   </devices>
Apr 14 03:10:11 user nova-compute[71323]: </domain>
Apr 14 03:10:11 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-196777223',display_name='tempest-ServerStableDeviceRescueTest-server-196777223',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-196777223',id=26,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7w1V2h51K5WeC0lNrY7+q4q2XqyPZ1TB9opotq+DNhc9r2qQzgCUPc3xsU2/ON8yD/igMXO0IAbOHg83C68rdtapd1/KHmCO4Nj4cZtEPuSMjanMb8ZhuWuQX+Z2NrHw==',key_name='tempest-keypair-1447416444',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='82eb4fbf55f04294a3d97800142bdca9',ramdisk_id='',reservation_id='r-pniwrgja',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1642713032',owner_user_name='tempest-ServerStableDeviceRescueTest-1642713032-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3c683fe5636b49858857033797ff8e68',uuid=8a67985b-1938-4b7d-b486-13f6811611f3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converting VIF {"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG os_vif [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2d081ef4-97, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2d081ef4-97, col_values=(('external_ids', {'iface-id': '2d081ef4-9756-4571-b28c-0d4582c58f0a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:af:d1:f9', 'vm-uuid': '8a67985b-1938-4b7d-b486-13f6811611f3'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:11 user nova-compute[71323]: INFO os_vif [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97')
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No VIF found with MAC fa:16:3e:af:d1:f9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updated VIF entry in instance network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0d1b52b4-184b-4dd5-99f8-244f955d1cf6 req-037914c0-7d68-4898-b2cc-e85e6d8ea61f service nova] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:12 user nova-compute[71323]: INFO nova.compute.manager [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attaching volume d036730b-83f8-4181-bc64-2c80ee0e312f to /dev/vdb
Apr 14 03:10:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:12 user nova-compute[71323]: WARNING nova.compute.manager [req-550bbf57-4468-4873-aa8a-1273a2b68d3b req-b5bfe544-1a17-435f-8296-b12d0d0c503c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state building and task_state spawning.
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.utils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa3075f4-9fe1-4f2b-a23e-4c59886233fd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee1cb38b-8c23-4d78-833b-1842625d368e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[857718b8-7009-4b0d-bd6f-aa5e1f8b9ac7]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[05129b70-983f-4b1e-87c9-f30b05aa0d60]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:12 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:10:12 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[390fb4e7-2a77-4de3-98b7-d961c41dac9b]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG os_brick.utils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:12 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating existing volume attachment record: 35c493ab-857a-4662-8b80-53150558a58b {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:10:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:14 user nova-compute[71323]: WARNING nova.compute.manager [req-ba68b2ed-063a-46b6-9f13-8f57ca8f0a11 req-9265bdad-940d-4c97-922c-2de52d618ad0 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state building and task_state spawning.
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441799.9160886, 7a1e2563-0dc3-4cc9-aa61-7cb321f28622 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:14 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] VM Stopped (Lifecycle Event)
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441814.933263, 8a67985b-1938-4b7d-b486-13f6811611f3 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Resumed (Lifecycle Event)
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aab536af-9570-4aa2-a69c-c5f24eb8387f None None] [instance: 7a1e2563-0dc3-4cc9-aa61-7cb321f28622] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance spawned successfully.
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:10:14 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441814.9350362, 8a67985b-1938-4b7d-b486-13f6811611f3 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:14 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Started (Lifecycle Event)
Apr 14 03:10:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:10:15 user nova-compute[71323]: INFO nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Took 6.48 seconds to spawn the instance on the hypervisor.
Apr 14 03:10:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:15 user nova-compute[71323]: INFO nova.compute.manager [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Took 7.76 seconds to build instance.
Apr 14 03:10:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ba3dd626-9044-467c-aefa-25d1f6f5fb02 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.875s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TEzqAxBi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:17 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:17 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:17 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260" returned: 21 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[68d427b4-2d4b-4fcd-848d-5234d973663d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:10:17 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1a3108e-4b3c-4a7f-83aa-023b0716c922]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f] added
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e478efee-e074-452e-8913-89626667c5e0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[705e0af7-8383-42ed-91aa-95fd415b2553]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TEzqAxBi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TEzqAxBi" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a4c74ae-11a2-4638-b3f7-9f36ada8fda4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'TEzqAxBi'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9047216-32c6-416d-baf0-43e42935dd0c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6faf515f-8084-4e35-91df-6d318ead7e82]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[109da6ac-9f96-41f7-b32a-44d8a5f6fb0d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260]
Apr 14 03:10:17 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8bff6329-27ba-4ffd-be27-f12b987b3237]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dafe8ea-f9d8-400d-ab0b-02f01c12a605]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:17 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.333s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] HCTL ('34', '-', '-', 0) found on session 30 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdb66f66-bcd3-4873-aad6-677af7e3b72a]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Searching for a device in session 30 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:10:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TEzqAxBi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:18 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Ignoring supplied device name: /dev/vdb
Apr 14 03:10:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.065s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.369s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== connect_volume: return (1370ms) {'type': 'block', 'scsi_wwn': '26131663861383139', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26131663861383139', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:18 user nova-compute[71323]: INFO nova.compute.manager [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Attaching volume a2998d0d-9ff4-4d46-a62f-30f7df32be1f to /dev/vdb
Apr 14 03:10:18 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:18 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:10:18 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:18 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 03:10:18 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:18 user nova-compute[71323]:   <serial>d036730b-83f8-4181-bc64-2c80ee0e312f</serial>
Apr 14 03:10:18 user nova-compute[71323]: </disk>
Apr 14 03:10:18 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.utils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0baf7591-f1f8-4ca2-a4e3-13873142214c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[25ddad6a-4c4c-4f47-82bb-d26da001354e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a7c7bec-39ee-4e0f-b435-f3038aeaba1e]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5f62ed8-99b0-48e5-bad5-da63c317c607]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:19 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb92d4e7-38cb-483e-9d35-2217b5b4ab77]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG os_brick.utils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== get_connector_properties: return (131ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updating existing volume attachment record: 57342732-1f05-437e-8055-2bd938640127 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No VIF found with MAC fa:16:3e:5d:ff:d3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b7c0da23-79d5-45f0-9878-273d642d3aac tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.782s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:21 user nova-compute[71323]: INFO nova.compute.manager [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Unrescuing
Apr 14 03:10:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'flavor' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:22 user nova-compute[71323]: WARNING nova.compute.manager [req-e6e4103b-f262-4e35-837a-536605389aca req-7864e27c-ee1d-47d2-b67b-8146e10e8d49 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:10:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.058s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:23 user nova-compute[71323]: INFO nova.compute.manager [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attaching volume 4ab737c6-db1d-4122-beea-9f6938803b8a to /dev/vdc
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.utils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance destroyed successfully.
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f24231e0-d1b3-464d-a3b8-3a3b05a33542]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2265f111-0246-4231-b27d-412251cf625b]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[802fdad5-28a2-4369-9456-5c65a48bbc88]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a657a95-ee0c-4718-910a-5fdc31dc76b7]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:23 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[307dc839-65c4-4703-a5e7-cabc994d8f11]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.utils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== get_connector_properties: return (145ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating existing volume attachment record: 8f2f9ba1-eeec-4e0d-9cfc-fbf0e2df3ee7 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZzhfkMEd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:23 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:23 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:23 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7339b2b5-9cb5-42a8-807b-77870bc0404f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:10:23 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae2c8176-c5c6-4985-90b1-271c8c119906]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f] added
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18c48063-9db0-45e2-bcf0-33ce24b240bb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef1c7ae0-8daa-42c2-bce4-b8d72dc26e0e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ZzhfkMEd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ZzhfkMEd" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1aaa6aa4-904c-4494-a223-f7a045aae8af]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ZzhfkMEd'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[436dc16c-97ea-4bce-98b7-00c2849934ad]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fb57cfa-1ab6-427f-8d26-5ac467597e9b]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --login" returned: 0 in 0.081s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[169e76ce-2382-450d-9b30-d2e7ff3a0226]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260]
Apr 14 03:10:24 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6142529a-16e8-4ce4-833e-81213942f46e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bd0a198-2e3f-495e-9677-8faff5026662]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.342s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] HCTL ('35', '-', '-', 0) found on session 31 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce1bd94-7eae-4594-8ecf-8804b25ed6ef]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Searching for a device in session 31 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZzhfkMEd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:24 user nova-compute[71323]: WARNING nova.compute.manager [req-087e0de9-8044-4d29-a5ef-19a9506144e3 req-146c8dc7-1be0-4ae9-aed0-68b258aa2bbe service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:10:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.374s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== connect_volume: return (1375ms) {'type': 'block', 'scsi_wwn': '26637326361333532', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26637326361333532', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:10:25 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:25 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:10:25 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:25 user nova-compute[71323]:   <serial>a2998d0d-9ff4-4d46-a62f-30f7df32be1f</serial>
Apr 14 03:10:25 user nova-compute[71323]: </disk>
Apr 14 03:10:25 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No VIF found with MAC fa:16:3e:17:63:b9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9dd36333-ea8d-4201-9359-feeada9c4a70 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.042s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:26 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:10:26 user nova-compute[71323]: INFO nova.compute.claims [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Claim successful on node user
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:27 user nova-compute[71323]: WARNING nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:27 user nova-compute[71323]: WARNING nova.compute.manager [req-37b646c3-b373-4a37-9513-2f7c05c5323a req-58e6e455-f0fd-45de-b80c-7420e78dab16 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.417s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:10:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 4e47adee-bd94-4655-9bbd-ed59c5478531 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441827.3318884, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:27 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Resumed (Lifecycle Event)
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d92e85f1-255b-47cb-90c5-790370a9eb00 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.policy [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b2facbf385e8402c86706cc28d4707af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1318728a20064d49b9c1b2f89175a364', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:10:27 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441827.333468, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:27 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Started (Lifecycle Event)
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:10:27 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Creating image(s)
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "/opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "/opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "/opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.191s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk 1073741824" returned: 0 in 0.251s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.448s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.164s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Checking if we can resize image /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Successfully created port: 6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Cannot resize image /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'migration_context' on Instance uuid 1840cf40-2228-4506-9a3f-0c9d29373533 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Ensure instance console log exists: /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DMJkXsZY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:28 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:28 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:28 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bb136c7-c2d8-42f9-ba2e-1ea8ce27590d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:10:28 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[60f804ee-8bca-49ab-b241-da9bb88280fd]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a] added
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[585ad287-2382-43e6-a24c-c1a47d312f4f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9b134c7-6f07-4b0c-aba0-6252152f0484]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DMJkXsZY {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DMJkXsZY" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0ebdbc4-eefc-4810-8c32-9461ed59bd8d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DMJkXsZY'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[697e59e4-49fd-4868-b327-a6ca42d3940a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[01f2282a-dc3b-4593-80b0-9d59816180d2]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --login" returned: 0 in 0.107s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4533bf77-f482-41a1-829e-4ef0b5599bac]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260]
Apr 14 03:10:28 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.039s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[51486cec-6745-4ec4-8e72-e1dd4e37df7f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2c182ec-3f51-407b-b7da-9195b53db32a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:28 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:28 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.400s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] HCTL ('36', '-', '-', 0) found on session 32 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:10:28 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b08dd06a-1b30-4d9c-a523-5c156d3d329e]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Searching for a device in session 32 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DMJkXsZY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Successfully updated port: 6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquired lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-changed-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Refreshing instance network info cache due to event network-changed-6c925a4c-0f5a-4711-92b2-7440a5170256. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] Acquiring lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Updating instance_info_cache with network_info: [{"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Releasing lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Instance network_info: |[{"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] Acquired lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.neutron [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Refreshing network info cache for port 6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Start _get_guest_xml network_info=[{"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:10:29 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:29 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-604635131',display_name='tempest-VolumesActionsTest-instance-604635131',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-604635131',id=27,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-5rybi00u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:27Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=1840cf40-2228-4506-9a3f-0c9d29373533,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'pci_devices' on Instance uuid 1840cf40-2228-4506-9a3f-0c9d29373533 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:10:29 user nova-compute[71323]:   <uuid>1840cf40-2228-4506-9a3f-0c9d29373533</uuid>
Apr 14 03:10:29 user nova-compute[71323]:   <name>instance-0000001b</name>
Apr 14 03:10:29 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:10:29 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:10:29 user nova-compute[71323]:   <metadata>
Apr 14 03:10:29 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:10:29 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:name>tempest-VolumesActionsTest-instance-604635131</nova:name>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:10:29</nova:creationTime>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:10:29 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:10:29 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:owner>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:user uuid="b2facbf385e8402c86706cc28d4707af">tempest-VolumesActionsTest-121883605-project-member</nova:user>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:project uuid="1318728a20064d49b9c1b2f89175a364">tempest-VolumesActionsTest-121883605</nova:project>
Apr 14 03:10:29 user nova-compute[71323]:       </nova:owner>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:10:29 user nova-compute[71323]:       <nova:ports>
Apr 14 03:10:29 user nova-compute[71323]:         <nova:port uuid="6c925a4c-0f5a-4711-92b2-7440a5170256">
Apr 14 03:10:29 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Apr 14 03:10:29 user nova-compute[71323]:         </nova:port>
Apr 14 03:10:29 user nova-compute[71323]:       </nova:ports>
Apr 14 03:10:29 user nova-compute[71323]:     </nova:instance>
Apr 14 03:10:29 user nova-compute[71323]:   </metadata>
Apr 14 03:10:29 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:10:29 user nova-compute[71323]:     <system>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="serial">1840cf40-2228-4506-9a3f-0c9d29373533</entry>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="uuid">1840cf40-2228-4506-9a3f-0c9d29373533</entry>
Apr 14 03:10:29 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:10:29 user nova-compute[71323]:     </system>
Apr 14 03:10:29 user nova-compute[71323]:   </sysinfo>
Apr 14 03:10:29 user nova-compute[71323]:   <os>
Apr 14 03:10:29 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:10:29 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:10:29 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:10:29 user nova-compute[71323]:   </os>
Apr 14 03:10:29 user nova-compute[71323]:   <features>
Apr 14 03:10:29 user nova-compute[71323]:     <acpi/>
Apr 14 03:10:29 user nova-compute[71323]:     <apic/>
Apr 14 03:10:29 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:10:29 user nova-compute[71323]:   </features>
Apr 14 03:10:29 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:10:29 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:10:29 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:10:29 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:10:29 user nova-compute[71323]:   </clock>
Apr 14 03:10:29 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:10:29 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:10:29 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:10:29 user nova-compute[71323]:   </cpu>
Apr 14 03:10:29 user nova-compute[71323]:   <devices>
Apr 14 03:10:29 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:10:29 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:10:29 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk"/>
Apr 14 03:10:29 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:10:29 user nova-compute[71323]:     </disk>
Apr 14 03:10:29 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:10:29 user nova-compute[71323]:       <mac address="fa:16:3e:72:42:11"/>
Apr 14 03:10:29 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:29 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:10:29 user nova-compute[71323]:       <target dev="tap6c925a4c-0f"/>
Apr 14 03:10:29 user nova-compute[71323]:     </interface>
Apr 14 03:10:29 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:10:29 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/console.log" append="off"/>
Apr 14 03:10:29 user nova-compute[71323]:     </serial>
Apr 14 03:10:29 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:10:29 user nova-compute[71323]:     <video>
Apr 14 03:10:29 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:29 user nova-compute[71323]:     </video>
Apr 14 03:10:29 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:10:29 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:10:29 user nova-compute[71323]:     </rng>
Apr 14 03:10:29 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:10:29 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:10:29 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:10:29 user nova-compute[71323]:     </memballoon>
Apr 14 03:10:29 user nova-compute[71323]:   </devices>
Apr 14 03:10:29 user nova-compute[71323]: </domain>
Apr 14 03:10:29 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-604635131',display_name='tempest-VolumesActionsTest-instance-604635131',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-604635131',id=27,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-5rybi00u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:27Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=1840cf40-2228-4506-9a3f-0c9d29373533,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG os_vif [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6c925a4c-0f, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6c925a4c-0f, col_values=(('external_ids', {'iface-id': '6c925a4c-0f5a-4711-92b2-7440a5170256', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:72:42:11', 'vm-uuid': '1840cf40-2228-4506-9a3f-0c9d29373533'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:29 user nova-compute[71323]: INFO os_vif [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f')
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] No VIF found with MAC fa:16:3e:72:42:11, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.527s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== connect_volume: return (1529ms) {'type': 'block', 'scsi_wwn': '23464393730363966', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23464393730363966', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:10:30 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:30 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:10:30 user nova-compute[71323]:   <target dev="vdc" bus="virtio"/>
Apr 14 03:10:30 user nova-compute[71323]:   <serial>4ab737c6-db1d-4122-beea-9f6938803b8a</serial>
Apr 14 03:10:30 user nova-compute[71323]: </disk>
Apr 14 03:10:30 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG nova.network.neutron [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Updated VIF entry in instance network info cache for port 6c925a4c-0f5a-4711-92b2-7440a5170256. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG nova.network.neutron [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Updating instance_info_cache with network_info: [{"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4863620e-d38b-4bef-ab55-2166804784ec req-0bd96d18-a697-4bb7-ae7e-ed8249da3261 service nova] Releasing lock "refresh_cache-1840cf40-2228-4506-9a3f-0c9d29373533" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] No VIF found with MAC fa:16:3e:5d:ff:d3, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c688884e-23bd-4dde-8a6a-3aae4f38d758 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.799s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] No waiting events found dispatching network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:31 user nova-compute[71323]: WARNING nova.compute.manager [req-ba8a4b17-8cae-4ca1-9b8c-da6838606b59 req-2dcc9a49-e57b-4897-af4d-e619ad3280ae service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received unexpected event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 for instance with vm_state building and task_state spawning.
Apr 14 03:10:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:32 user nova-compute[71323]: INFO nova.compute.manager [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Detaching volume d036730b-83f8-4181-bc64-2c80ee0e312f
Apr 14 03:10:32 user nova-compute[71323]: INFO nova.virt.block_device [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attempting to driver detach volume d036730b-83f8-4181-bc64-2c80ee0e312f from mountpoint /dev/vdb
Apr 14 03:10:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Attempting to detach device vdb from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:10:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:32 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:32 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 03:10:32 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:32 user nova-compute[71323]:   <serial>d036730b-83f8-4181-bc64-2c80ee0e312f</serial>
Apr 14 03:10:32 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:10:32 user nova-compute[71323]: </disk>
Apr 14 03:10:32 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:32 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdb from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the persistent domain config.
Apr 14 03:10:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:10:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:32 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:32 user nova-compute[71323]:   <source dev="/dev/sdc"/>
Apr 14 03:10:32 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:32 user nova-compute[71323]:   <serial>d036730b-83f8-4181-bc64-2c80ee0e312f</serial>
Apr 14 03:10:32 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:10:32 user nova-compute[71323]: </disk>
Apr 14 03:10:32 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441833.38965, 9c4bbe1e-d297-40e2-95d5-d8ab381c302d => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:10:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdb from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the live domain config.
Apr 14 03:10:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TEzqAxBi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:33 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9fa7621-3214-4e06-8018-54ad29cc42d0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:10:33 user nova-compute[71323]: No targets found.
Apr 14 03:10:33 user nova-compute[71323]: iSNS:
Apr 14 03:10:33 user nova-compute[71323]: No targets found.
Apr 14 03:10:33 user nova-compute[71323]: STATIC:
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a
Apr 14 03:10:33 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:33 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:33 user nova-compute[71323]: FIRMWARE:
Apr 14 03:10:33 user nova-compute[71323]: No targets found.
Apr 14 03:10:33 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:10:33 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:10:33 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[238b8271-068b-4767-b81d-eae934fa14af]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efbc8923-466e-41c7-99c9-60046b426505]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:33 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [30] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:33 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:33 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae0192680>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[157017de-d2ce-4157-b0c7-4437a9443603]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0607a90c-a6af-483d-83a5-6befe5be10b3]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28ce798d-6825-4b95-ad2d-bad344d2827f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[affe1841-2eb5-4853-a39b-9d80f2dd19c1]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.155s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a80e999e-2946-498c-9f91-0dda244297d5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --logout" returned: 0 in 0.083s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac48779f-cd55-4e6e-aeeb-0b582ccdc1f2]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260]
Apr 14 03:10:33 user nova-compute[71323]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:33 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d036730b-83f8-4181-bc64-2c80ee0e312f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[57ef3673-b045-4884-b84a-2831604f5487]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.551s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== disconnect_volume: return (553ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] No waiting events found dispatching network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:34 user nova-compute[71323]: WARNING nova.compute.manager [req-a6e0df49-9e85-41df-b536-105bfe82d06d req-25897d14-3aca-4fe6-b85c-7030505235c9 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received unexpected event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 for instance with vm_state building and task_state spawning.
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441834.4970987, 1840cf40-2228-4506-9a3f-0c9d29373533 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] VM Resumed (Lifecycle Event)
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Instance spawned successfully.
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441834.4984932, 1840cf40-2228-4506-9a3f-0c9d29373533 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] VM Started (Lifecycle Event)
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:10:34 user nova-compute[71323]: INFO nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Took 7.13 seconds to spawn the instance on the hypervisor.
Apr 14 03:10:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:35 user nova-compute[71323]: INFO nova.compute.manager [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Took 8.28 seconds to build instance.
Apr 14 03:10:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2710d8dd-5d95-4d9b-b13e-0bb810f38291 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.390s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a5c32c2-c42b-4282-b664-6c1ff43a4803 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.350s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:36 user nova-compute[71323]: INFO nova.compute.manager [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Detaching volume 4ab737c6-db1d-4122-beea-9f6938803b8a
Apr 14 03:10:36 user nova-compute[71323]: INFO nova.virt.block_device [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Attempting to driver detach volume 4ab737c6-db1d-4122-beea-9f6938803b8a from mountpoint /dev/vdc
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Attempting to detach device vdc from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:36 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:36 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:10:36 user nova-compute[71323]:   <target dev="vdc" bus="virtio"/>
Apr 14 03:10:36 user nova-compute[71323]:   <serial>4ab737c6-db1d-4122-beea-9f6938803b8a</serial>
Apr 14 03:10:36 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 03:10:36 user nova-compute[71323]: </disk>
Apr 14 03:10:36 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdc from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the persistent domain config.
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:36 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:36 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:10:36 user nova-compute[71323]:   <target dev="vdc" bus="virtio"/>
Apr 14 03:10:36 user nova-compute[71323]:   <serial>4ab737c6-db1d-4122-beea-9f6938803b8a</serial>
Apr 14 03:10:36 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Apr 14 03:10:36 user nova-compute[71323]: </disk>
Apr 14 03:10:36 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441836.8260746, 9c4bbe1e-d297-40e2-95d5-d8ab381c302d => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:10:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully detached device vdc from instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d from the live domain config.
Apr 14 03:10:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DMJkXsZY', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[68d7ca26-d307-408e-97b1-c8b8c2307e06]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:10:36 user nova-compute[71323]: No targets found.
Apr 14 03:10:36 user nova-compute[71323]: iSNS:
Apr 14 03:10:36 user nova-compute[71323]: No targets found.
Apr 14 03:10:36 user nova-compute[71323]: STATIC:
Apr 14 03:10:36 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:10:36 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:36 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:36 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:10:36 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:36 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:36 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:10:36 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:36 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:36 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f
Apr 14 03:10:36 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:36 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:36 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a
Apr 14 03:10:36 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:36 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:36 user nova-compute[71323]: FIRMWARE:
Apr 14 03:10:36 user nova-compute[71323]: No targets found.
Apr 14 03:10:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:10:36 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:10:36 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[19c03af6-6897-4f87-bcac-27f9cae8f6db]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cbe4637-c482-4879-96bc-922fff7cf5fb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:36 user nova-compute[71323]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a (non-flash)
Apr 14 03:10:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae0193f40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b7234d6-5e69-4bbf-90ad-665c4d766ed0]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[23264e73-58b1-48d1-aa56-fb1734b35d70]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc277ec6-ab9c-4ab9-a877-e1ea69768fc1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:10:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[94ca9fc8-bf78-487f-a312-06a5d1975848]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[299d32d6-b40e-4bf7-bba3-2c1e69c14217]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --logout" returned: 0 in 0.075s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb240222-b520-4b7d-9252-88e3dec4aac9]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260]
Apr 14 03:10:37 user nova-compute[71323]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ab737c6-db1d-4122-beea-9f6938803b8a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[edc26463-a2ee-478b-a77f-0550e03ed644]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.337s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] <== disconnect_volume: return (339ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:10:38 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'flavor' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b45712cc-23c9-45f0-85d7-122bede1cee3 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.811s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:39 user nova-compute[71323]: INFO nova.compute.manager [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Terminating instance
Apr 14 03:10:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-unplugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] No waiting events found dispatching network-vif-unplugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-16de160b-fe08-4954-ac55-04beba4a71cf req-c0bacfd6-4936-4ce3-8f68-8ac286522a99 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-unplugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Instance destroyed successfully.
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lazy-loading 'resources' on Instance uuid 9c4bbe1e-d297-40e2-95d5-d8ab381c302d {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1987903443',display_name='tempest-AttachVolumeTestJSON-server-1987903443',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1987903443',id=24,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDFjzX0+40uO37CfqkOBSmbMWwyV5wsfc6L7vAuDWYejkbH4JLkWAIz1Wn0uLtB1OjrTaShOAhE1miwOi7ekeCgSgaeKQp/3FeYMl2Pn1H8x1Gn7ZQIusaMOU4fzl1tfrQ==',key_name='tempest-keypair-1327512481',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='130665008cf648cf87467ae2777a9170',ramdisk_id='',reservation_id='r-quz7070b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-975876676',owner_user_name='tempest-AttachVolumeTestJSON-975876676-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d6d78d4991b5493e8893034e91ed407b',uuid=9c4bbe1e-d297-40e2-95d5-d8ab381c302d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converting VIF {"id": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "address": "fa:16:3e:5d:ff:d3", "network": {"id": "079db793-280b-4ba4-8733-eec1c8591b70", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-979312196-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "130665008cf648cf87467ae2777a9170", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapcf6c0efd-f0", "ovs_interfaceid": "cf6c0efd-f0d7-4a17-837f-eefb2b1a5731", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG os_vif [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapcf6c0efd-f0, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:10:40 user nova-compute[71323]: INFO os_vif [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:5d:ff:d3,bridge_name='br-int',has_traffic_filtering=True,id=cf6c0efd-f0d7-4a17-837f-eefb2b1a5731,network=Network(079db793-280b-4ba4-8733-eec1c8591b70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapcf6c0efd-f0')
Apr 14 03:10:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Deleting instance files /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d_del
Apr 14 03:10:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Deletion of /opt/stack/data/nova/instances/9c4bbe1e-d297-40e2-95d5-d8ab381c302d_del complete
Apr 14 03:10:40 user nova-compute[71323]: INFO nova.compute.manager [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Took 0.90 seconds to destroy the instance on the hypervisor.
Apr 14 03:10:40 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:10:40 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:40 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Took 0.84 seconds to deallocate network for instance.
Apr 14 03:10:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-b3c9b5d6-7de0-424b-82fa-4fd9595589c7 req-9697d9f4-ddc2-4f67-84dd-bba77bb62703 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-deleted-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.375s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:41 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Deleted allocations for instance 9c4bbe1e-d297-40e2-95d5-d8ab381c302d
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2799c90d-992a-4936-bf80-75ea6c7f30e0 tempest-AttachVolumeTestJSON-975876676 tempest-AttachVolumeTestJSON-975876676-project-member] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.309s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] Acquiring lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] Lock "9c4bbe1e-d297-40e2-95d5-d8ab381c302d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] No waiting events found dispatching network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:41 user nova-compute[71323]: WARNING nova.compute.manager [req-459a2f2f-b933-4184-b435-4f3180d7dcf9 req-875db15c-eae2-4f2c-b356-8309a431cb76 service nova] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Received unexpected event network-vif-plugged-cf6c0efd-f0d7-4a17-837f-eefb2b1a5731 for instance with vm_state deleted and task_state None.
Apr 14 03:10:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:46 user nova-compute[71323]: INFO nova.compute.manager [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Detaching volume a2998d0d-9ff4-4d46-a62f-30f7df32be1f
Apr 14 03:10:46 user nova-compute[71323]: INFO nova.virt.block_device [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Attempting to driver detach volume a2998d0d-9ff4-4d46-a62f-30f7df32be1f from mountpoint /dev/vdb
Apr 14 03:10:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Attempting to detach device vdb from instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:10:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:46 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:46 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:10:46 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:46 user nova-compute[71323]:   <serial>a2998d0d-9ff4-4d46-a62f-30f7df32be1f</serial>
Apr 14 03:10:46 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:10:46 user nova-compute[71323]: </disk>
Apr 14 03:10:46 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully detached device vdb from instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a from the persistent domain config.
Apr 14 03:10:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:10:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:10:46 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:46 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:10:46 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:10:46 user nova-compute[71323]:   <serial>a2998d0d-9ff4-4d46-a62f-30f7df32be1f</serial>
Apr 14 03:10:46 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:10:46 user nova-compute[71323]: </disk>
Apr 14 03:10:46 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441847.1767485, e1493ecb-e6a8-48c5-abf3-d3cbc092736a => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:10:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully detached device vdb from instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a from the live domain config.
Apr 14 03:10:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZzhfkMEd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:47 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f9ea82c-7dc8-44a4-a0fe-6d79e6abb8f8]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:10:47 user nova-compute[71323]: No targets found.
Apr 14 03:10:47 user nova-compute[71323]: iSNS:
Apr 14 03:10:47 user nova-compute[71323]: No targets found.
Apr 14 03:10:47 user nova-compute[71323]: STATIC:
Apr 14 03:10:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:10:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:10:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:10:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:47 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f
Apr 14 03:10:47 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:47 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:47 user nova-compute[71323]: FIRMWARE:
Apr 14 03:10:47 user nova-compute[71323]: No targets found.
Apr 14 03:10:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:10:47 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:10:47 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae5f61bc-51e3-420c-8e6e-291bf92415fc]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a163ed74-7f6d-43aa-8ee7-d1dafc16a337]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:47 user nova-compute[71323]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f (non-flash)
Apr 14 03:10:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b004d6560>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d21f8e18-5c43-4cec-a2a8-49909534fa01]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8510678f-57b9-44ec-8d83-60ba38019fab]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecee6a5e-0344-4fc6-a9c7-b6b155a45b07]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fefa0714-8fe8-4b0f-90de-9bcc35572304]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1321e164-9d2b-459d-a55c-656ad381d271]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[04f2906b-e1c8-4a8b-a50f-b2bd791a6d07]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260]
Apr 14 03:10:47 user nova-compute[71323]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f46eeca-ff02-4da8-a238-60a3baf6fbb0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.323s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== disconnect_volume: return (323ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b05a1aab-89ba-40e9-a0ee-e33539da9e76 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.642s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.544s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:48 user nova-compute[71323]: INFO nova.compute.manager [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Terminating instance
Apr 14 03:10:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-unplugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] No waiting events found dispatching network-vif-unplugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c4c3aba-5d8c-4cf8-83d9-feab4fad89e9 req-a0675d04-c612-430f-98e4-ffbebfa81fe9 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-unplugged-136dada7-395e-410e-8eee-03e4d82dc01a for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Instance destroyed successfully.
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'resources' on Instance uuid e1493ecb-e6a8-48c5-abf3-d3cbc092736a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:08:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-2069905968',display_name='tempest-VolumesSnapshotTestJSON-instance-2069905968',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-2069905968',id=25,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-b6bvow3h',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:38Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=e1493ecb-e6a8-48c5-abf3-d3cbc092736a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "136dada7-395e-410e-8eee-03e4d82dc01a", "address": "fa:16:3e:17:63:b9", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap136dada7-39", "ovs_interfaceid": "136dada7-395e-410e-8eee-03e4d82dc01a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_vif [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap136dada7-39, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:10:49 user nova-compute[71323]: INFO os_vif [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:17:63:b9,bridge_name='br-int',has_traffic_filtering=True,id=136dada7-395e-410e-8eee-03e4d82dc01a,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap136dada7-39')
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ZzhfkMEd', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:49 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[44aa3c03-ecd5-4799-8c13-7087f116054d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:10:49 user nova-compute[71323]: No targets found.
Apr 14 03:10:49 user nova-compute[71323]: iSNS:
Apr 14 03:10:49 user nova-compute[71323]: No targets found.
Apr 14 03:10:49 user nova-compute[71323]: STATIC:
Apr 14 03:10:49 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:10:49 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:49 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:49 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:10:49 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:49 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:49 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:10:49 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:10:49 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:10:49 user nova-compute[71323]: FIRMWARE:
Apr 14 03:10:49 user nova-compute[71323]: No targets found.
Apr 14 03:10:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:10:49 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:10:49 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a2998d0d-9ff4-4d46-a62f-30f7df32be1f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4ebafa2-ed6c-4c9a-abce-f5fc7589ff66]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbf9f500-d424-4590-897e-f37aa0424581]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:49 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:49 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:49 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:49 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:49 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae01b2b00>, {}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Disconnecting from: [] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.081s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== disconnect_volume: return (82ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Deleting instance files /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a_del
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Deletion of /opt/stack/data/nova/instances/e1493ecb-e6a8-48c5-abf3-d3cbc092736a_del complete
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.compute.manager [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Took 0.94 seconds to destroy the instance on the hypervisor.
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Took 0.45 seconds to deallocate network for instance.
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-8b507c25-beab-4386-89b5-e43a13ac090c req-b375ac20-2062-4bca-8963-02b3765eb5e2 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-deleted-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.compute.manager [req-8b507c25-beab-4386-89b5-e43a13ac090c req-b375ac20-2062-4bca-8963-02b3765eb5e2 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Neutron deleted interface 136dada7-395e-410e-8eee-03e4d82dc01a; detaching it from the instance and deleting it from the info cache
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-8b507c25-beab-4386-89b5-e43a13ac090c req-b375ac20-2062-4bca-8963-02b3765eb5e2 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:49 user nova-compute[71323]: WARNING nova.volume.cinder [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Attachment 57342732-1f05-437e-8055-2bd938640127 does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = 57342732-1f05-437e-8055-2bd938640127. (HTTP 404) (Request-ID: req-6736c33f-4b38-4f41-b0ff-a5e9ea494885)
Apr 14 03:10:49 user nova-compute[71323]: INFO nova.compute.manager [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Took 0.02 seconds to detach 1 volumes for instance.
Apr 14 03:10:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-8b507c25-beab-4386-89b5-e43a13ac090c req-b375ac20-2062-4bca-8963-02b3765eb5e2 service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Detach interface failed, port_id=136dada7-395e-410e-8eee-03e4d82dc01a, reason: Instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.290s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:50 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Deleted allocations for instance e1493ecb-e6a8-48c5-abf3-d3cbc092736a
Apr 14 03:10:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6ba778d4-8092-4bb5-83c3-b233fbc50417 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.897s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] Acquiring lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] Lock "e1493ecb-e6a8-48c5-abf3-d3cbc092736a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] No waiting events found dispatching network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:50 user nova-compute[71323]: WARNING nova.compute.manager [req-a1875c42-b5ad-4bdd-adae-dc64705d5bdd req-d186f96c-3989-4901-8303-4134f7f60f0c service nova] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Received unexpected event network-vif-plugged-136dada7-395e-410e-8eee-03e4d82dc01a for instance with vm_state deleted and task_state None.
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:10:51 user nova-compute[71323]: INFO nova.compute.claims [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Claim successful on node user
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.374s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:10:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:10:51 user nova-compute[71323]: INFO nova.virt.block_device [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Booting with volume c6fbfee2-70fa-477a-a761-8d092061e691 at /dev/vda
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.policy [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.utils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[37c02139-bf3c-4e73-aae6-f3d5d17db998]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf7d0264-5707-49d7-9667-03ff1ffa43c1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fcf1ac2-1e5d-44b6-af45-8f3fd9dea29d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7877736d-015e-4f0c-80eb-9ee302ab186c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:10:51 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:10:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de9f63f6-f6fb-4792-a891-810b952e27fc]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG os_brick.utils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:51 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating existing volume attachment record: 3f82a8e5-bedd-4384-b5b2-6a269185f6a0 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:10:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Successfully created port: 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Successfully updated port: 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing instance network info cache due to event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] Acquiring lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating instance_info_cache with network_info: [{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Instance network_info: |[{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] Acquired lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:10:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:10:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updated VIF entry in instance network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:10:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating instance_info_cache with network_info: [{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:10:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2e4d8779-3e56-44cf-b963-ac878bc76632 req-0bdd3c5a-84ac-4552-a951-c4d577b69b9d service nova] Releasing lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:10:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:55 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441840.0156548, 9c4bbe1e-d297-40e2-95d5-d8ab381c302d => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:10:55 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] VM Stopped (Lifecycle Event)
Apr 14 03:10:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8c31a9fc-868d-46f5-9b4e-53f55db2cf10 None None] [instance: 9c4bbe1e-d297-40e2-95d5-d8ab381c302d] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:10:56 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Creating image(s)
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Ensure instance console log exists: /opt/stack/data/nova/instances/7f542934-641f-405f-811c-3601a3be389b/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Start _get_guest_xml network_info=[{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '3f82a8e5-bedd-4384-b5b2-6a269185f6a0', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '7f542934-641f-405f-811c-3601a3be389b', 'attached_at': '', 'detached_at': '', 'volume_id': 'c6fbfee2-70fa-477a-a761-8d092061e691', 'serial': 'c6fbfee2-70fa-477a-a761-8d092061e691'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:10:56 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:56 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:10:56 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.004s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:56 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:10:56 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b12bde7-69d4-4b9e-bd94-4e579233617a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:10:56 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d8a681a-d4b3-40bd-b2d2-3f41bfc73302]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691] added
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a3aea7c-e524-4dd0-87c8-690433fd0912]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d723e51-5dbe-4d6f-9e0c-ae9619b77d6a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tekamKoO {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tekamKoO" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[36267ebc-5e72-4f9c-b430-c5fe5a80b7ca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tekamKoO'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bab80b50-749d-49a7-878b-9159091422b3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[19728998-1895-40bb-9621-f0f6a3531102]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad861e97-7f88-43d5-b2fe-812aa3cae213]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]
Apr 14 03:10:56 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a869e0c-88c6-4aaf-bc2a-46556d38ea4e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc9717ee-57e3-4127-bf37-996eda51a00a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:10:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.309s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('34', '-', '-', 0) found on session 33 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[014b08fb-214a-4689-874e-ef4d81f29dc8]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:10:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 33 and hctl ['34', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 33 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.342s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1345ms) {'type': 'block', 'scsi_wwn': '23433326361643831', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23433326361643831', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1780392212',display_name='tempest-TestVolumeBootPattern-server-1780392212',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1780392212',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-33ksjg22',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:52Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7f542934-641f-405f-811c-3601a3be389b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.objects.instance [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid 7f542934-641f-405f-811c-3601a3be389b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:10:57 user nova-compute[71323]:   <uuid>7f542934-641f-405f-811c-3601a3be389b</uuid>
Apr 14 03:10:57 user nova-compute[71323]:   <name>instance-0000001c</name>
Apr 14 03:10:57 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:10:57 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:10:57 user nova-compute[71323]:   <metadata>
Apr 14 03:10:57 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:10:57 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:10:57 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-server-1780392212</nova:name>
Apr 14 03:10:57 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:10:56</nova:creationTime>
Apr 14 03:10:57 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:10:57 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:10:57 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:10:57 user nova-compute[71323]:       <nova:owner>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 03:10:57 user nova-compute[71323]:       </nova:owner>
Apr 14 03:10:57 user nova-compute[71323]:       <nova:ports>
Apr 14 03:10:57 user nova-compute[71323]:         <nova:port uuid="1ccf0327-517f-45c5-a995-1b3a7fcfdaf9">
Apr 14 03:10:57 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:10:57 user nova-compute[71323]:         </nova:port>
Apr 14 03:10:57 user nova-compute[71323]:       </nova:ports>
Apr 14 03:10:57 user nova-compute[71323]:     </nova:instance>
Apr 14 03:10:57 user nova-compute[71323]:   </metadata>
Apr 14 03:10:57 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:10:57 user nova-compute[71323]:     <system>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="serial">7f542934-641f-405f-811c-3601a3be389b</entry>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="uuid">7f542934-641f-405f-811c-3601a3be389b</entry>
Apr 14 03:10:57 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:10:57 user nova-compute[71323]:     </system>
Apr 14 03:10:57 user nova-compute[71323]:   </sysinfo>
Apr 14 03:10:57 user nova-compute[71323]:   <os>
Apr 14 03:10:57 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:10:57 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:10:57 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:10:57 user nova-compute[71323]:   </os>
Apr 14 03:10:57 user nova-compute[71323]:   <features>
Apr 14 03:10:57 user nova-compute[71323]:     <acpi/>
Apr 14 03:10:57 user nova-compute[71323]:     <apic/>
Apr 14 03:10:57 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:10:57 user nova-compute[71323]:   </features>
Apr 14 03:10:57 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:10:57 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:10:57 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:10:57 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:10:57 user nova-compute[71323]:   </clock>
Apr 14 03:10:57 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:10:57 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:10:57 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:10:57 user nova-compute[71323]:   </cpu>
Apr 14 03:10:57 user nova-compute[71323]:   <devices>
Apr 14 03:10:57 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:10:57 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:10:57 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:10:57 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:10:57 user nova-compute[71323]:       <serial>c6fbfee2-70fa-477a-a761-8d092061e691</serial>
Apr 14 03:10:57 user nova-compute[71323]:     </disk>
Apr 14 03:10:57 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:10:57 user nova-compute[71323]:       <mac address="fa:16:3e:b5:5a:1f"/>
Apr 14 03:10:57 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:57 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:10:57 user nova-compute[71323]:       <target dev="tap1ccf0327-51"/>
Apr 14 03:10:57 user nova-compute[71323]:     </interface>
Apr 14 03:10:57 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:10:57 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/7f542934-641f-405f-811c-3601a3be389b/console.log" append="off"/>
Apr 14 03:10:57 user nova-compute[71323]:     </serial>
Apr 14 03:10:57 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:10:57 user nova-compute[71323]:     <video>
Apr 14 03:10:57 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:10:57 user nova-compute[71323]:     </video>
Apr 14 03:10:57 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:10:57 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:10:57 user nova-compute[71323]:     </rng>
Apr 14 03:10:57 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:10:57 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:10:57 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:10:57 user nova-compute[71323]:     </memballoon>
Apr 14 03:10:57 user nova-compute[71323]:   </devices>
Apr 14 03:10:57 user nova-compute[71323]: </domain>
Apr 14 03:10:57 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1780392212',display_name='tempest-TestVolumeBootPattern-server-1780392212',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1780392212',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-33ksjg22',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:10:52Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7f542934-641f-405f-811c-3601a3be389b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG os_vif [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1ccf0327-51, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap1ccf0327-51, col_values=(('external_ids', {'iface-id': '1ccf0327-517f-45c5-a995-1b3a7fcfdaf9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b5:5a:1f', 'vm-uuid': '7f542934-641f-405f-811c-3601a3be389b'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:57 user nova-compute[71323]: INFO os_vif [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51')
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:b5:5a:1f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] No waiting events found dispatching network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:10:59 user nova-compute[71323]: WARNING nova.compute.manager [req-d561bc62-3602-433a-aed5-7f1a8540e421 req-92b7c43b-9ce4-4173-921a-8f4ef03edd18 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received unexpected event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 for instance with vm_state building and task_state spawning.
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:10:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441861.1653981, 7f542934-641f-405f-811c-3601a3be389b => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] VM Resumed (Lifecycle Event)
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] Instance spawned successfully.
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441861.166699, 7f542934-641f-405f-811c-3601a3be389b => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] VM Started (Lifecycle Event)
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Took 4.98 seconds to spawn the instance on the hypervisor.
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] No waiting events found dispatching network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:01 user nova-compute[71323]: WARNING nova.compute.manager [req-6091bb27-a9a2-472c-b599-4cc9fb48896d req-0302ccee-8ea2-4af3-a0e9-73cc9a3a0508 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received unexpected event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 for instance with vm_state building and task_state spawning.
Apr 14 03:11:01 user nova-compute[71323]: INFO nova.compute.manager [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Took 10.28 seconds to build instance.
Apr 14 03:11:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aac587f1-4dac-46f8-94ba-8ef0716b3dfe tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.377s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:11:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441849.0566328, e1493ecb-e6a8-48c5-abf3-d3cbc092736a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:04 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] VM Stopped (Lifecycle Event)
Apr 14 03:11:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ce58e688-0442-4505-a72b-a8be7b408ec2 None None] [instance: e1493ecb-e6a8-48c5-abf3-d3cbc092736a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.170s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4/disk --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a/disk.rescue --force-share --output=json" returned: 0 in 0.127s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:06 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:11:06 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:11:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7694MB free_disk=26.424114227294922GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128, 'DISK_GB': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1840cf40-2228-4506-9a3f-0c9d29373533 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7f542934-641f-405f-811c-3601a3be389b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.462s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:11:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:11:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:11:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:11:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:11:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:11:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:30 user nova-compute[71323]: INFO nova.compute.manager [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Terminating instance
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-442d61e8-4d77-4929-a0ad-e8cd2c965ac7 req-82762498-6f33-4089-9651-9b26399f7fb8 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-unplugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:11:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Instance destroyed successfully.
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T02:57:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-1395552505',display_name='tempest-ServerActionsTestJSON-server-1052167823',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-1395552505',id=5,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:55Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-jj7ocist',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:08:55Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=6bb082bc-0c74-4149-9bd0-ceb09d7764d4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "address": "fa:16:3e:af:bb:dd", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap04b5c1f1-89", "ovs_interfaceid": "04b5c1f1-8988-4a34-b97a-ad0cecd12b71", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_vif [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap04b5c1f1-89, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:11:30 user nova-compute[71323]: INFO os_vif [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:bb:dd,bridge_name='br-int',has_traffic_filtering=True,id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap04b5c1f1-89')
Apr 14 03:11:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'rCsYcqle', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:11:30 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[88f6264a-886a-4768-8c3e-fc39a6d2310b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:11:30 user nova-compute[71323]: No targets found.
Apr 14 03:11:30 user nova-compute[71323]: iSNS:
Apr 14 03:11:30 user nova-compute[71323]: No targets found.
Apr 14 03:11:30 user nova-compute[71323]: STATIC:
Apr 14 03:11:30 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:11:30 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:11:30 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:11:30 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:11:30 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:11:30 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:11:30 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:11:30 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:11:30 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:11:30 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983
Apr 14 03:11:30 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:11:30 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:11:30 user nova-compute[71323]: FIRMWARE:
Apr 14 03:11:30 user nova-compute[71323]: No targets found.
Apr 14 03:11:30 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:11:30 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:11:30 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[37822d80-cc34-4dd4-ab99-ddac800a3b0e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[396d1390-6ca8-413f-a6f6-f6749195be6a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:11:31 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 (non-flash)
Apr 14 03:11:31 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:11:31 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae01b0940>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c8e55b5-36ba-442d-a6a1-249755a34707]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2456f7a-cbe9-4ed5-9bbf-1ba637065a5d]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6fdba4f-9db4-419f-af02-a35a0f1ef7c5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.046s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[96c3dbb8-29aa-458d-a00e-396ded6a3bb8]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93bd3332-a70f-4b13-b3c7-65872d5edf6c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[900244a7-814e-4993-9acc-c646140366b6]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260]
Apr 14 03:11:31 user nova-compute[71323]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983, portal: 172.16.0.220,3260] successful.
Apr 14 03:11:31 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:44f3cbc9-fd81-4185-aa74-0896739af983 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdbde8aa-befb-48cd-aa37-e2bbaae5f845]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.321s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (322ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:11:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Deleting instance files /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_del
Apr 14 03:11:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Deletion of /opt/stack/data/nova/instances/6bb082bc-0c74-4149-9bd0-ceb09d7764d4_del complete
Apr 14 03:11:31 user nova-compute[71323]: INFO nova.compute.manager [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Took 1.29 seconds to destroy the instance on the hypervisor.
Apr 14 03:11:31 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:11:31 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:32 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Took 0.91 seconds to deallocate network for instance.
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-129b92d7-61b2-4fc7-94ee-78bd46315ff2 req-c192eecb-30f9-44c0-8649-c9dd8bdae502 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-deleted-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:32 user nova-compute[71323]: INFO nova.compute.manager [req-129b92d7-61b2-4fc7-94ee-78bd46315ff2 req-c192eecb-30f9-44c0-8649-c9dd8bdae502 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Neutron deleted interface 04b5c1f1-8988-4a34-b97a-ad0cecd12b71; detaching it from the instance and deleting it from the info cache
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.network.neutron [req-129b92d7-61b2-4fc7-94ee-78bd46315ff2 req-c192eecb-30f9-44c0-8649-c9dd8bdae502 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-129b92d7-61b2-4fc7-94ee-78bd46315ff2 req-c192eecb-30f9-44c0-8649-c9dd8bdae502 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Detach interface failed, port_id=04b5c1f1-8988-4a34-b97a-ad0cecd12b71, reason: Instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] Acquiring lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] No waiting events found dispatching network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:32 user nova-compute[71323]: WARNING nova.compute.manager [req-b5ab8b28-df32-4fe9-84aa-25a2373f00bd req-cd89a5b1-fcfe-4b9d-ae24-bdad587d0c99 service nova] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Received unexpected event network-vif-plugged-04b5c1f1-8988-4a34-b97a-ad0cecd12b71 for instance with vm_state active and task_state deleting.
Apr 14 03:11:33 user nova-compute[71323]: INFO nova.compute.manager [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Took 0.89 seconds to detach 1 volumes for instance.
Apr 14 03:11:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:33 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:11:33 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:11:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.471s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:33 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Deleted allocations for instance 6bb082bc-0c74-4149-9bd0-ceb09d7764d4
Apr 14 03:11:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5fac981a-162e-453b-87d0-6cfc008f053d tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "6bb082bc-0c74-4149-9bd0-ceb09d7764d4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.815s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:40 user nova-compute[71323]: INFO nova.compute.manager [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Unrescuing
Apr 14 03:11:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'flavor' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:41 user nova-compute[71323]: WARNING nova.compute.manager [req-fb9cf409-d540-45e8-9e14-7216c75d1e85 req-31b02940-8bd3-4b8f-a4bc-35bd3ef925be service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance destroyed successfully.
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:43 user nova-compute[71323]: WARNING nova.compute.manager [req-c666cc07-c350-4896-89de-686fdedaf1bc req-9f6a85ce-d5cb-42f8-a4d2-f1826350bf8e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:11:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441905.1135523, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Resumed (Lifecycle Event)
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3dd041a5-a5e3-457b-9ff2-33992268856c tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441905.117038, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Started (Lifecycle Event)
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:45 user nova-compute[71323]: WARNING nova.compute.manager [req-03d50509-4303-4148-99e7-9dfe31100d37 req-0192336d-4b41-4845-81d3-a2e0099381c8 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state None.
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441890.883188, 6bb082bc-0c74-4149-9bd0-ceb09d7764d4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:45 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] VM Stopped (Lifecycle Event)
Apr 14 03:11:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-48f45ae2-41e4-4430-b982-c761264253e5 None None] [instance: 6bb082bc-0c74-4149-9bd0-ceb09d7764d4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:11:51 user nova-compute[71323]: INFO nova.compute.claims [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Claim successful on node user
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.374s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:11:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:11:52 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.policy [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0128a733bd6a4120addba5eda1cc05fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4549f6080dcf4c739ac03ef643801fbc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:11:52 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Creating image(s)
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "/opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "/opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "/opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk 1073741824" returned: 0 in 0.058s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.208s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Checking if we can resize image /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Successfully created port: 6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Cannot resize image /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.objects.instance [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'migration_context' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Ensure instance console log exists: /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Successfully updated port: 6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquired lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-changed-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Refreshing instance network info cache due to event network-changed-6f05389c-b6d4-40ce-afa3-06e781a35ba5. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] Acquiring lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Updating instance_info_cache with network_info: [{"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Releasing lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Instance network_info: |[{"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] Acquired lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Refreshing network info cache for port 6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Start _get_guest_xml network_info=[{"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:11:53 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:11:53 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:11:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1353831390',display_name='tempest-VolumesSnapshotTestJSON-instance-1353831390',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1353831390',id=29,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-lni03qpo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:11:52Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=0708e26d-9d20-4cb0-9b8c-76249f74f0a2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'pci_devices' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:11:53 user nova-compute[71323]:   <uuid>0708e26d-9d20-4cb0-9b8c-76249f74f0a2</uuid>
Apr 14 03:11:53 user nova-compute[71323]:   <name>instance-0000001d</name>
Apr 14 03:11:53 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:11:53 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:11:53 user nova-compute[71323]:   <metadata>
Apr 14 03:11:53 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:11:53 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-1353831390</nova:name>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:11:53</nova:creationTime>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:11:53 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:11:53 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:owner>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:user uuid="0128a733bd6a4120addba5eda1cc05fb">tempest-VolumesSnapshotTestJSON-423841644-project-member</nova:user>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:project uuid="4549f6080dcf4c739ac03ef643801fbc">tempest-VolumesSnapshotTestJSON-423841644</nova:project>
Apr 14 03:11:53 user nova-compute[71323]:       </nova:owner>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:11:53 user nova-compute[71323]:       <nova:ports>
Apr 14 03:11:53 user nova-compute[71323]:         <nova:port uuid="6f05389c-b6d4-40ce-afa3-06e781a35ba5">
Apr 14 03:11:53 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:11:53 user nova-compute[71323]:         </nova:port>
Apr 14 03:11:53 user nova-compute[71323]:       </nova:ports>
Apr 14 03:11:53 user nova-compute[71323]:     </nova:instance>
Apr 14 03:11:53 user nova-compute[71323]:   </metadata>
Apr 14 03:11:53 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:11:53 user nova-compute[71323]:     <system>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="serial">0708e26d-9d20-4cb0-9b8c-76249f74f0a2</entry>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="uuid">0708e26d-9d20-4cb0-9b8c-76249f74f0a2</entry>
Apr 14 03:11:53 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:11:53 user nova-compute[71323]:     </system>
Apr 14 03:11:53 user nova-compute[71323]:   </sysinfo>
Apr 14 03:11:53 user nova-compute[71323]:   <os>
Apr 14 03:11:53 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:11:53 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:11:53 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:11:53 user nova-compute[71323]:   </os>
Apr 14 03:11:53 user nova-compute[71323]:   <features>
Apr 14 03:11:53 user nova-compute[71323]:     <acpi/>
Apr 14 03:11:53 user nova-compute[71323]:     <apic/>
Apr 14 03:11:53 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:11:53 user nova-compute[71323]:   </features>
Apr 14 03:11:53 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:11:53 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:11:53 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:11:53 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:11:53 user nova-compute[71323]:   </clock>
Apr 14 03:11:53 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:11:53 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:11:53 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:11:53 user nova-compute[71323]:   </cpu>
Apr 14 03:11:53 user nova-compute[71323]:   <devices>
Apr 14 03:11:53 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:11:53 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:11:53 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk"/>
Apr 14 03:11:53 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:11:53 user nova-compute[71323]:     </disk>
Apr 14 03:11:53 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:11:53 user nova-compute[71323]:       <mac address="fa:16:3e:90:9c:13"/>
Apr 14 03:11:53 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:11:53 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:11:53 user nova-compute[71323]:       <target dev="tap6f05389c-b6"/>
Apr 14 03:11:53 user nova-compute[71323]:     </interface>
Apr 14 03:11:53 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:11:53 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/console.log" append="off"/>
Apr 14 03:11:53 user nova-compute[71323]:     </serial>
Apr 14 03:11:53 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:11:53 user nova-compute[71323]:     <video>
Apr 14 03:11:53 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:11:53 user nova-compute[71323]:     </video>
Apr 14 03:11:53 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:11:53 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:11:53 user nova-compute[71323]:     </rng>
Apr 14 03:11:53 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:11:53 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:11:53 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:11:53 user nova-compute[71323]:     </memballoon>
Apr 14 03:11:53 user nova-compute[71323]:   </devices>
Apr 14 03:11:53 user nova-compute[71323]: </domain>
Apr 14 03:11:53 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:11:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1353831390',display_name='tempest-VolumesSnapshotTestJSON-instance-1353831390',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1353831390',id=29,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-lni03qpo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:11:52Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=0708e26d-9d20-4cb0-9b8c-76249f74f0a2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG os_vif [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6f05389c-b6, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6f05389c-b6, col_values=(('external_ids', {'iface-id': '6f05389c-b6d4-40ce-afa3-06e781a35ba5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:90:9c:13', 'vm-uuid': '0708e26d-9d20-4cb0-9b8c-76249f74f0a2'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:11:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:53 user nova-compute[71323]: INFO os_vif [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6')
Apr 14 03:11:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:11:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No VIF found with MAC fa:16:3e:90:9c:13, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:11:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Updated VIF entry in instance network info cache for port 6f05389c-b6d4-40ce-afa3-06e781a35ba5. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:11:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Updating instance_info_cache with network_info: [{"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3e72b0e6-2368-4dc7-ac4c-a63573a71955 req-51fda451-0e0f-40b0-b318-b205d1f12872 service nova] Releasing lock "refresh_cache-0708e26d-9d20-4cb0-9b8c-76249f74f0a2" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] No waiting events found dispatching network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:55 user nova-compute[71323]: WARNING nova.compute.manager [req-3b9a09d8-2645-442a-82e5-20e003830444 req-1f99ac62-9e87-4fc7-898f-ffbcd9c76b23 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received unexpected event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 for instance with vm_state building and task_state spawning.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] No waiting events found dispatching network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:11:57 user nova-compute[71323]: WARNING nova.compute.manager [req-35cd6664-a9b1-4524-ba1e-79c4e7ce8359 req-43780a10-ff12-4b11-966d-48e78dfdd77f service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received unexpected event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 for instance with vm_state building and task_state spawning.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441917.6431684, 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] VM Resumed (Lifecycle Event)
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Instance spawned successfully.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441917.6446047, 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] VM Started (Lifecycle Event)
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:11:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Took 5.62 seconds to spawn the instance on the hypervisor.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:57 user nova-compute[71323]: INFO nova.compute.manager [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Took 6.31 seconds to build instance.
Apr 14 03:11:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aa7a4c74-3dfc-4ece-b627-361d18ed3f98 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.411s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG nova.compute.manager [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing instance network info cache due to event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updated VIF entry in instance network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG nova.network.neutron [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:11:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0e090c34-e8dc-45fd-906b-77ba89345e0b req-13ca3962-514d-42d9-9fbb-5819fd535a3e service nova] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:11:59 user nova-compute[71323]: INFO nova.compute.manager [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] instance snapshotting
Apr 14 03:11:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Beginning live snapshot process
Apr 14 03:11:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json -f qcow2" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:11:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json -f qcow2" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1.delta 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1.delta 1073741824" returned: 0 in 0.056s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 03:12:00 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:12:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:12:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:12:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 03:12:01 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:12:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1.delta /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1.delta /opt/stack/data/nova/instances/snapshots/tmpj158hzcu/24b9057c4aac460785b8526d217e15c1" returned: 0 in 0.481s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:02 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Snapshot extracted, beginning image upload
Apr 14 03:12:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Snapshot image upload complete
Apr 14 03:12:04 user nova-compute[71323]: INFO nova.compute.manager [None req-560fbb1f-0326-4d8d-9983-a9c92e3540ff tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Took 5.04 seconds to snapshot the instance on the hypervisor.
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:12:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:12:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:07 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:07 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7785MB free_disk=26.3963623046875GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 1840cf40-2228-4506-9a3f-0c9d29373533 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7f542934-641f-405f-811c-3601a3be389b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:12:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:12:08 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:12:08 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:12:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:12:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.426s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:12:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing instance network info cache due to event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:12:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:14 user nova-compute[71323]: INFO nova.compute.manager [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Detaching volume 05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:12:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:14 user nova-compute[71323]: INFO nova.virt.block_device [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Attempting to driver detach volume 05d362e8-1aaf-4bbb-8a9f-9f153920e00c from mountpoint /dev/vdb
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Attempting to detach device vdb from instance 4e47adee-bd94-4655-9bbd-ed59c5478531 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:12:14 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:12:14 user nova-compute[71323]:   <source dev="/dev/sdh"/>
Apr 14 03:12:14 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:12:14 user nova-compute[71323]:   <serial>05d362e8-1aaf-4bbb-8a9f-9f153920e00c</serial>
Apr 14 03:12:14 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:12:14 user nova-compute[71323]: </disk>
Apr 14 03:12:14 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:12:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully detached device vdb from instance 4e47adee-bd94-4655-9bbd-ed59c5478531 from the persistent domain config.
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 4e47adee-bd94-4655-9bbd-ed59c5478531 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:12:14 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:12:14 user nova-compute[71323]:   <source dev="/dev/sdh"/>
Apr 14 03:12:14 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:12:14 user nova-compute[71323]:   <serial>05d362e8-1aaf-4bbb-8a9f-9f153920e00c</serial>
Apr 14 03:12:14 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:12:14 user nova-compute[71323]: </disk>
Apr 14 03:12:14 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updated VIF entry in instance network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-355fcf83-00bf-4ab3-8eb3-283e1fef6d8d req-cdc18c60-9388-404c-abc8-b66a3dc23f74 service nova] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681441934.6785896, 4e47adee-bd94-4655-9bbd-ed59c5478531 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:12:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully detached device vdb from instance 4e47adee-bd94-4655-9bbd-ed59c5478531 from the live domain config.
Apr 14 03:12:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'KcHRlyjP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:12:14 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0c32ab6-f6e1-480c-95f3-8cdb7ee3baf9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:12:14 user nova-compute[71323]: No targets found.
Apr 14 03:12:14 user nova-compute[71323]: iSNS:
Apr 14 03:12:14 user nova-compute[71323]: No targets found.
Apr 14 03:12:14 user nova-compute[71323]: STATIC:
Apr 14 03:12:14 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c
Apr 14 03:12:14 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:12:14 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:12:14 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:12:14 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:12:14 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:12:14 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:12:14 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:12:14 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:12:14 user nova-compute[71323]: FIRMWARE:
Apr 14 03:12:14 user nova-compute[71323]: No targets found.
Apr 14 03:12:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:12:14 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:12:14 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed57dcef-272a-432d-9f64-2089c420e2f3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[526752fe-be96-4cbd-934b-25cc66572494]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:14 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:12:14 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:14 user nova-compute[71323]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c (non-flash)
Apr 14 03:12:14 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b007df490>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c'): ({'sdh'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Removing single pathed devices sdh {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5aaa0fa3-372f-40d8-936d-4cab32a40b9f]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7bc17ee-b13a-4fc4-924a-d86ab61cc2af]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Flushing IO for device /dev/sdh {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4ae107f-88c7-4097-9df6-b830e18bf57e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.045s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[404c7498-c42a-4410-bb51-df9cf0b0cfbb]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] SCSI volumes sdh have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7faa9cff-c290-456a-a6c2-4865300bbfc1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1de97b6b-a4ff-4859-8556-27582f2c756b]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260]
Apr 14 03:12:14 user nova-compute[71323]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c, portal: 172.16.0.220,3260] successful.
Apr 14 03:12:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:05d362e8-1aaf-4bbb-8a9f-9f153920e00c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[235201ff-98a4-4749-b79f-f3e9ea5a7a9c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.324s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] <== disconnect_volume: return (325ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing instance network info cache due to event network-changed-4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] Acquiring lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] Acquired lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:15 user nova-compute[71323]: DEBUG nova.network.neutron [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Refreshing network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:12:16 user nova-compute[71323]: DEBUG nova.objects.instance [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'flavor' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:12:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-da18d952-1e93-4b08-bbf1-dcbfd9b84297 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.024s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:16 user nova-compute[71323]: DEBUG nova.network.neutron [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updated VIF entry in instance network info cache for port 4773c5d2-1520-4fc1-ae46-f83a883cfd1b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:12:16 user nova-compute[71323]: DEBUG nova.network.neutron [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [{"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ce63d06d-812b-4e0c-a7fc-406b5f8c189a req-9a3235a2-b7c2-4520-a85c-aa2de76c4cd9 service nova] Releasing lock "refresh_cache-4e47adee-bd94-4655-9bbd-ed59c5478531" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:17 user nova-compute[71323]: INFO nova.compute.manager [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Terminating instance
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-unplugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] No waiting events found dispatching network-vif-unplugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-d1a7ce39-94de-4b47-8dba-7d8f9454645b req-7ebc9195-53ac-4e44-ab1e-68318397a270 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-unplugged-6c925a4c-0f5a-4711-92b2-7440a5170256 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:12:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Instance destroyed successfully.
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'resources' on Instance uuid 1840cf40-2228-4506-9a3f-0c9d29373533 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-604635131',display_name='tempest-VolumesActionsTest-instance-604635131',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-604635131',id=27,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:10:34Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-5rybi00u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:10:35Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=1840cf40-2228-4506-9a3f-0c9d29373533,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "6c925a4c-0f5a-4711-92b2-7440a5170256", "address": "fa:16:3e:72:42:11", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6c925a4c-0f", "ovs_interfaceid": "6c925a4c-0f5a-4711-92b2-7440a5170256", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG os_vif [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6c925a4c-0f, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:17 user nova-compute[71323]: INFO os_vif [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:72:42:11,bridge_name='br-int',has_traffic_filtering=True,id=6c925a4c-0f5a-4711-92b2-7440a5170256,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6c925a4c-0f')
Apr 14 03:12:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Deleting instance files /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533_del
Apr 14 03:12:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Deletion of /opt/stack/data/nova/instances/1840cf40-2228-4506-9a3f-0c9d29373533_del complete
Apr 14 03:12:17 user nova-compute[71323]: INFO nova.compute.manager [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Took 0.85 seconds to destroy the instance on the hypervisor.
Apr 14 03:12:17 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:12:17 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:18 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Took 0.50 seconds to deallocate network for instance.
Apr 14 03:12:18 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b8bb0a0-f860-4660-a1f8-e381c5541438 req-c738abae-41bb-4430-a86d-02f56822f620 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-deleted-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:12:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.295s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:18 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Deleted allocations for instance 1840cf40-2228-4506-9a3f-0c9d29373533
Apr 14 03:12:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b3ec2471-15f5-4ace-b779-7758f3c40505 tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "1840cf40-2228-4506-9a3f-0c9d29373533" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.840s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] Acquiring lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] Lock "1840cf40-2228-4506-9a3f-0c9d29373533-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] No waiting events found dispatching network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:12:19 user nova-compute[71323]: WARNING nova.compute.manager [req-0ca9d075-4555-4d3a-9704-c6a265335ab5 req-b4c76b09-c5e3-43f4-b799-309a2a9607d4 service nova] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Received unexpected event network-vif-plugged-6c925a4c-0f5a-4711-92b2-7440a5170256 for instance with vm_state deleted and task_state None.
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:12:22 user nova-compute[71323]: INFO nova.compute.claims [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Claim successful on node user
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.419s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:12:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:12:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Creating image(s)
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "/opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "/opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk 1073741824" returned: 0 in 0.053s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.195s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.policy [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7dea23743ba945c4a9d539b8e277c653', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4075885e26cb4adaad24d3ae6217fe35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking if we can resize image /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Cannot resize image /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'migration_context' on Instance uuid 24d1359d-2dcd-46d7-90f3-3801df9be357 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Ensure instance console log exists: /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:23 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Successfully created port: 6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Successfully updated port: 6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing instance network info cache due to event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Instance network_info: |[{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.network.neutron [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:12:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Start _get_guest_xml network_info=[{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:12:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-207746006',display_name='tempest-ServerActionsTestJSON-server-207746006',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-207746006',id=30,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-4xkvk9c0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=24d1359d-2dcd-46d7-90f3-3801df9be357,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid 24d1359d-2dcd-46d7-90f3-3801df9be357 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:12:25 user nova-compute[71323]:   <uuid>24d1359d-2dcd-46d7-90f3-3801df9be357</uuid>
Apr 14 03:12:25 user nova-compute[71323]:   <name>instance-0000001e</name>
Apr 14 03:12:25 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:12:25 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:12:25 user nova-compute[71323]:   <metadata>
Apr 14 03:12:25 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:12:25 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-207746006</nova:name>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:12:25</nova:creationTime>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:12:25 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:12:25 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:owner>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:12:25 user nova-compute[71323]:       </nova:owner>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:12:25 user nova-compute[71323]:       <nova:ports>
Apr 14 03:12:25 user nova-compute[71323]:         <nova:port uuid="6ea4c00e-5d4b-4375-9b8d-60177a6ec526">
Apr 14 03:12:25 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:12:25 user nova-compute[71323]:         </nova:port>
Apr 14 03:12:25 user nova-compute[71323]:       </nova:ports>
Apr 14 03:12:25 user nova-compute[71323]:     </nova:instance>
Apr 14 03:12:25 user nova-compute[71323]:   </metadata>
Apr 14 03:12:25 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:12:25 user nova-compute[71323]:     <system>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="serial">24d1359d-2dcd-46d7-90f3-3801df9be357</entry>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="uuid">24d1359d-2dcd-46d7-90f3-3801df9be357</entry>
Apr 14 03:12:25 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:12:25 user nova-compute[71323]:     </system>
Apr 14 03:12:25 user nova-compute[71323]:   </sysinfo>
Apr 14 03:12:25 user nova-compute[71323]:   <os>
Apr 14 03:12:25 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:12:25 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:12:25 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:12:25 user nova-compute[71323]:   </os>
Apr 14 03:12:25 user nova-compute[71323]:   <features>
Apr 14 03:12:25 user nova-compute[71323]:     <acpi/>
Apr 14 03:12:25 user nova-compute[71323]:     <apic/>
Apr 14 03:12:25 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:12:25 user nova-compute[71323]:   </features>
Apr 14 03:12:25 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:12:25 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:12:25 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:12:25 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:12:25 user nova-compute[71323]:   </clock>
Apr 14 03:12:25 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:12:25 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:12:25 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:12:25 user nova-compute[71323]:   </cpu>
Apr 14 03:12:25 user nova-compute[71323]:   <devices>
Apr 14 03:12:25 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:12:25 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:12:25 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk"/>
Apr 14 03:12:25 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:12:25 user nova-compute[71323]:     </disk>
Apr 14 03:12:25 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:12:25 user nova-compute[71323]:       <mac address="fa:16:3e:23:9b:c6"/>
Apr 14 03:12:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:12:25 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:12:25 user nova-compute[71323]:       <target dev="tap6ea4c00e-5d"/>
Apr 14 03:12:25 user nova-compute[71323]:     </interface>
Apr 14 03:12:25 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:12:25 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/console.log" append="off"/>
Apr 14 03:12:25 user nova-compute[71323]:     </serial>
Apr 14 03:12:25 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:12:25 user nova-compute[71323]:     <video>
Apr 14 03:12:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:12:25 user nova-compute[71323]:     </video>
Apr 14 03:12:25 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:12:25 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:12:25 user nova-compute[71323]:     </rng>
Apr 14 03:12:25 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:12:25 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:12:25 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:12:25 user nova-compute[71323]:     </memballoon>
Apr 14 03:12:25 user nova-compute[71323]:   </devices>
Apr 14 03:12:25 user nova-compute[71323]: </domain>
Apr 14 03:12:25 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-207746006',display_name='tempest-ServerActionsTestJSON-server-207746006',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-207746006',id=30,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-4xkvk9c0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=24d1359d-2dcd-46d7-90f3-3801df9be357,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG os_vif [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6ea4c00e-5d, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap6ea4c00e-5d, col_values=(('external_ids', {'iface-id': '6ea4c00e-5d4b-4375-9b8d-60177a6ec526', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:23:9b:c6', 'vm-uuid': '24d1359d-2dcd-46d7-90f3-3801df9be357'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:25 user nova-compute[71323]: INFO os_vif [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d')
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:23:9b:c6, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updated VIF entry in instance network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG nova.network.neutron [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d2130da3-436f-486b-8625-7d76227697a4 req-cb925e25-0591-43c5-90cc-aaaab6b79def service nova] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:26 user nova-compute[71323]: DEBUG nova.compute.manager [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] No waiting events found dispatching network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:12:26 user nova-compute[71323]: WARNING nova.compute.manager [req-3c041257-6f24-410c-89b1-e7d807ad8c90 req-eccfdc88-4099-4fcd-8ab7-517bc1a10600 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received unexpected event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 for instance with vm_state building and task_state spawning.
Apr 14 03:12:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441948.5642767, 24d1359d-2dcd-46d7-90f3-3801df9be357 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] VM Resumed (Lifecycle Event)
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Instance spawned successfully.
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441948.565691, 24d1359d-2dcd-46d7-90f3-3801df9be357 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] VM Started (Lifecycle Event)
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Took 5.85 seconds to spawn the instance on the hypervisor.
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:12:28 user nova-compute[71323]: INFO nova.compute.manager [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Took 6.57 seconds to build instance.
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] No waiting events found dispatching network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:12:28 user nova-compute[71323]: WARNING nova.compute.manager [req-98d2f1e4-97cc-4def-be0d-763673364e1c req-8f3c5aad-3a62-4737-9d7a-09ecf8ed4be3 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received unexpected event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 for instance with vm_state active and task_state None.
Apr 14 03:12:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2ff94d75-f234-4cdf-bdfe-ad3adc79e2f3 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.669s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:32 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441937.8206208, 1840cf40-2228-4506-9a3f-0c9d29373533 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:12:32 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] VM Stopped (Lifecycle Event)
Apr 14 03:12:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e9b62a42-0ef3-4983-8900-a32c83619270 None None] [instance: 1840cf40-2228-4506-9a3f-0c9d29373533] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:12:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing instance network info cache due to event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:12:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] Acquiring lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] Acquired lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:12:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updated VIF entry in instance network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:12:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating instance_info_cache with network_info: [{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.57", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4b1b295a-0b06-41e1-98a9-b8ba2e2d72b5 req-bd2b2045-4049-4348-b7a3-e04c850bd458 service nova] Releasing lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:52 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:12:52 user nova-compute[71323]: INFO nova.compute.claims [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Claim successful on node user
Apr 14 03:12:52 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:12:53 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:12:53 user nova-compute[71323]: INFO nova.virt.block_device [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Booting with volume 0ae128f0-64ab-4413-bae2-15cd90212c53 at /dev/vda
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.policy [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '60e11b7ab2914209b23bf7a53791da64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d2e49e8e92e416faebf55e3d4e0e087', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.utils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ad42785-5aa1-4967-a4f5-c07b1065bbea]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[82f281c7-c41f-4713-a2ba-73ff83d71d06]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[046ac9e0-da3c-4641-9b51-8b0182d091ec]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f1f6775-f22c-474f-8d33-7c978b80ae5e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:12:53 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:12:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5583d6e-3554-4295-a10e-59be54fc09e3]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG os_brick.utils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating existing volume attachment record: b02ad57f-7445-460d-9b5a-d11efdcd54d4 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:12:53 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Successfully created port: 2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:12:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Successfully updated port: 2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquired lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing instance network info cache due to event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] Acquiring lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.network.neutron [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating instance_info_cache with network_info: [{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Releasing lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Instance network_info: |[{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] Acquired lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:12:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:12:56 user nova-compute[71323]: DEBUG nova.network.neutron [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updated VIF entry in instance network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:12:56 user nova-compute[71323]: DEBUG nova.network.neutron [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating instance_info_cache with network_info: [{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:12:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4519703a-f1b3-4c7d-98e6-f153d246dc06 req-d35c85af-2acb-4fa6-b086-76b42e2b2384 service nova] Releasing lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:12:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Creating image(s)
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Ensure instance console log exists: /opt/stack/data/nova/instances/5d0f51bd-b28d-4db1-be36-cec6cc92231e/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Start _get_guest_xml network_info=[{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'b02ad57f-7445-460d-9b5a-d11efdcd54d4', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tyovbWPn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '5d0f51bd-b28d-4db1-be36-cec6cc92231e', 'attached_at': '', 'detached_at': '', 'volume_id': '0ae128f0-64ab-4413-bae2-15cd90212c53', 'serial': '0ae128f0-64ab-4413-bae2-15cd90212c53'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:12:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:58 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tyovbWPn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:12:58 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:12:58 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:12:58 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[49dd6474-e0ea-4a48-a178-be0224412027]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:12:58 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21b65f27-cd72-47c9-8eda-b92f6eb0877a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53] added
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[74da1615-a8b4-4469-a6d3-a41bf29731d9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[50cb3838-51dd-4ba1-84a0-af716e2be726]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tyovbWPn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tyovbWPn" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2cd8683-0625-4fce-b4b8-e4f29ba855b5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tyovbWPn'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[47e78a7d-1060-40ad-8343-351591883cc8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9db6791-8a03-4118-8535-dd86e834d54a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2106d2a6-e0fe-43eb-8e64-916d3ba1cbc2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260]
Apr 14 03:12:58 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260] successful.
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fab17e82-a73e-4ac3-b0bd-591a0d3888bf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7450f5a-78d6-4e1c-a032-56de5b6149ae]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:12:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.301s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] HCTL ('33', '-', '-', 0) found on session 34 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f48d0931-294c-4842-9473-3c3aee7fca3e]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Searching for a device in session 34 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:12:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tyovbWPn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.334s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== connect_volume: return (1336ms) {'type': 'block', 'scsi_wwn': '23465636337363435', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23465636337363435', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1851758576',display_name='tempest-TestVolumeBootPattern-server-1851758576',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1851758576',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-engfvqb5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:12:53Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=5d0f51bd-b28d-4db1-be36-cec6cc92231e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'pci_devices' on Instance uuid 5d0f51bd-b28d-4db1-be36-cec6cc92231e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:12:59 user nova-compute[71323]:   <uuid>5d0f51bd-b28d-4db1-be36-cec6cc92231e</uuid>
Apr 14 03:12:59 user nova-compute[71323]:   <name>instance-0000001f</name>
Apr 14 03:12:59 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:12:59 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:12:59 user nova-compute[71323]:   <metadata>
Apr 14 03:12:59 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:12:59 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:12:59 user nova-compute[71323]:       <nova:name>tempest-TestVolumeBootPattern-server-1851758576</nova:name>
Apr 14 03:12:59 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:12:58</nova:creationTime>
Apr 14 03:12:59 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:12:59 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:12:59 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:12:59 user nova-compute[71323]:       <nova:owner>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:user uuid="60e11b7ab2914209b23bf7a53791da64">tempest-TestVolumeBootPattern-910285030-project-member</nova:user>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:project uuid="2d2e49e8e92e416faebf55e3d4e0e087">tempest-TestVolumeBootPattern-910285030</nova:project>
Apr 14 03:12:59 user nova-compute[71323]:       </nova:owner>
Apr 14 03:12:59 user nova-compute[71323]:       <nova:ports>
Apr 14 03:12:59 user nova-compute[71323]:         <nova:port uuid="2885c06a-4dea-452f-a76a-e6a12bb525cc">
Apr 14 03:12:59 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:12:59 user nova-compute[71323]:         </nova:port>
Apr 14 03:12:59 user nova-compute[71323]:       </nova:ports>
Apr 14 03:12:59 user nova-compute[71323]:     </nova:instance>
Apr 14 03:12:59 user nova-compute[71323]:   </metadata>
Apr 14 03:12:59 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:12:59 user nova-compute[71323]:     <system>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="serial">5d0f51bd-b28d-4db1-be36-cec6cc92231e</entry>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="uuid">5d0f51bd-b28d-4db1-be36-cec6cc92231e</entry>
Apr 14 03:12:59 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:12:59 user nova-compute[71323]:     </system>
Apr 14 03:12:59 user nova-compute[71323]:   </sysinfo>
Apr 14 03:12:59 user nova-compute[71323]:   <os>
Apr 14 03:12:59 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:12:59 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:12:59 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:12:59 user nova-compute[71323]:   </os>
Apr 14 03:12:59 user nova-compute[71323]:   <features>
Apr 14 03:12:59 user nova-compute[71323]:     <acpi/>
Apr 14 03:12:59 user nova-compute[71323]:     <apic/>
Apr 14 03:12:59 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:12:59 user nova-compute[71323]:   </features>
Apr 14 03:12:59 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:12:59 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:12:59 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:12:59 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:12:59 user nova-compute[71323]:   </clock>
Apr 14 03:12:59 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:12:59 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:12:59 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:12:59 user nova-compute[71323]:   </cpu>
Apr 14 03:12:59 user nova-compute[71323]:   <devices>
Apr 14 03:12:59 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:12:59 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:12:59 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:12:59 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:12:59 user nova-compute[71323]:       <serial>0ae128f0-64ab-4413-bae2-15cd90212c53</serial>
Apr 14 03:12:59 user nova-compute[71323]:     </disk>
Apr 14 03:12:59 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:12:59 user nova-compute[71323]:       <mac address="fa:16:3e:22:99:71"/>
Apr 14 03:12:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:12:59 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:12:59 user nova-compute[71323]:       <target dev="tap2885c06a-4d"/>
Apr 14 03:12:59 user nova-compute[71323]:     </interface>
Apr 14 03:12:59 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:12:59 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/5d0f51bd-b28d-4db1-be36-cec6cc92231e/console.log" append="off"/>
Apr 14 03:12:59 user nova-compute[71323]:     </serial>
Apr 14 03:12:59 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:12:59 user nova-compute[71323]:     <video>
Apr 14 03:12:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:12:59 user nova-compute[71323]:     </video>
Apr 14 03:12:59 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:12:59 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:12:59 user nova-compute[71323]:     </rng>
Apr 14 03:12:59 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:12:59 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:12:59 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:12:59 user nova-compute[71323]:     </memballoon>
Apr 14 03:12:59 user nova-compute[71323]:   </devices>
Apr 14 03:12:59 user nova-compute[71323]: </domain>
Apr 14 03:12:59 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1851758576',display_name='tempest-TestVolumeBootPattern-server-1851758576',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1851758576',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-engfvqb5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:12:53Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=5d0f51bd-b28d-4db1-be36-cec6cc92231e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG os_vif [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2885c06a-4d, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2885c06a-4d, col_values=(('external_ids', {'iface-id': '2885c06a-4dea-452f-a76a-e6a12bb525cc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:22:99:71', 'vm-uuid': '5d0f51bd-b28d-4db1-be36-cec6cc92231e'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:12:59 user nova-compute[71323]: INFO os_vif [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d')
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] No VIF found with MAC fa:16:3e:22:99:71, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:12:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] No waiting events found dispatching network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:01 user nova-compute[71323]: WARNING nova.compute.manager [req-13d6a822-2cea-437b-876b-a8f5c3e5fb35 req-ad123f65-1cc0-4d9d-8785-74dda117af9f service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received unexpected event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc for instance with vm_state building and task_state spawning.
Apr 14 03:13:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:13:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441983.138351, 5d0f51bd-b28d-4db1-be36-cec6cc92231e => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] VM Resumed (Lifecycle Event)
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Instance spawned successfully.
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441983.141813, 5d0f51bd-b28d-4db1-be36-cec6cc92231e => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] VM Started (Lifecycle Event)
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Took 5.08 seconds to spawn the instance on the hypervisor.
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] No waiting events found dispatching network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:03 user nova-compute[71323]: WARNING nova.compute.manager [req-167127c9-ae26-4966-b521-1501b53b65f7 req-d4f22ed8-c7ef-409e-9af7-887dc0a05b6b service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received unexpected event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc for instance with vm_state building and task_state spawning.
Apr 14 03:13:03 user nova-compute[71323]: INFO nova.compute.manager [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Took 10.72 seconds to build instance.
Apr 14 03:13:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-425dbef7-e950-41a7-98ea-4facba282d02 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.822s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:04 user nova-compute[71323]: INFO nova.compute.manager [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Terminating instance
Apr 14 03:13:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-d3ebfbe4-afb0-4a74-ac93-9795371f3bcc req-3dbd7f70-3f92-41e3-9d99-b3647bb22bb9 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-unplugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Instance destroyed successfully.
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid 4e47adee-bd94-4655-9bbd-ed59c5478531 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:06:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-413600185',display_name='tempest-ServerRescueNegativeTestJSON-server-413600185',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-413600185',id=20,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNRt1vkGxga6d2lnbq6r7KuWDr3P9XxrdKyfybFxOJEVdZ9m8h/yMxKRYhp7IzJglBQvcA3Veeq6pJJUHOvucLytIWZV1lh6jcIrnJ6ba4rOplP1XVeraWzy0zv10BSYng==',key_name='tempest-keypair-158606901',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:08:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-jnxbele2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:10:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d5e8150f1502427fb3148f20354d212e',uuid=4e47adee-bd94-4655-9bbd-ed59c5478531,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "address": "fa:16:3e:3f:e5:e9", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.163", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4773c5d2-15", "ovs_interfaceid": "4773c5d2-1520-4fc1-ae46-f83a883cfd1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG os_vif [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4773c5d2-15, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:13:05 user nova-compute[71323]: INFO os_vif [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:3f:e5:e9,bridge_name='br-int',has_traffic_filtering=True,id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4773c5d2-15')
Apr 14 03:13:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Deleting instance files /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531_del
Apr 14 03:13:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Deletion of /opt/stack/data/nova/instances/4e47adee-bd94-4655-9bbd-ed59c5478531_del complete
Apr 14 03:13:05 user nova-compute[71323]: INFO nova.compute.manager [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Took 1.12 seconds to destroy the instance on the hypervisor.
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Error from libvirt while getting description of instance-00000014: [Error Code 42] Domain not found: no domain with matching uuid '4e47adee-bd94-4655-9bbd-ed59c5478531' (instance-00000014): libvirt.libvirtError: Domain not found: no domain with matching uuid '4e47adee-bd94-4655-9bbd-ed59c5478531' (instance-00000014)
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5450388-57e5-4402-9dbd-7d72f09ebebd req-1bcc59f1-c07e-460e-a127-fcb3777ae0f7 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-deleted-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:06 user nova-compute[71323]: INFO nova.compute.manager [req-b5450388-57e5-4402-9dbd-7d72f09ebebd req-1bcc59f1-c07e-460e-a127-fcb3777ae0f7 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Neutron deleted interface 4773c5d2-1520-4fc1-ae46-f83a883cfd1b; detaching it from the instance and deleting it from the info cache
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.network.neutron [req-b5450388-57e5-4402-9dbd-7d72f09ebebd req-1bcc59f1-c07e-460e-a127-fcb3777ae0f7 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:06 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Took 1.10 seconds to deallocate network for instance.
Apr 14 03:13:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-b5450388-57e5-4402-9dbd-7d72f09ebebd req-1bcc59f1-c07e-460e-a127-fcb3777ae0f7 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Detach interface failed, port_id=4773c5d2-1520-4fc1-ae46-f83a883cfd1b, reason: Instance 4e47adee-bd94-4655-9bbd-ed59c5478531 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.181s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk.rescue --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:06 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] Acquiring lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] No waiting events found dispatching network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:07 user nova-compute[71323]: WARNING nova.compute.manager [req-861bb8b1-b639-4b69-a4fb-27b763fdb9a1 req-461d9cd6-0761-4128-aaa9-ea5bca61a600 service nova] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Received unexpected event network-vif-plugged-4773c5d2-1520-4fc1-ae46-f83a883cfd1b for instance with vm_state deleted and task_state None.
Apr 14 03:13:07 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.496s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:07 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Deleted allocations for instance 4e47adee-bd94-4655-9bbd-ed59c5478531
Apr 14 03:13:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-8ed2ebff-beb6-4ac7-a1fe-51590515c420 tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "4e47adee-bd94-4655-9bbd-ed59c5478531" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.013s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:08 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:08 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7842MB free_disk=26.363216400146484GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ac032176-63bc-457c-ad53-5b4a854eec34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7f542934-641f-405f-811c-3601a3be389b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5d0f51bd-b28d-4db1-be36-cec6cc92231e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d309dc50-d3b0-47a4-b263-a67ae96e2f5b has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1689}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1664MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:13:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.511s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.159s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:13:09 user nova-compute[71323]: INFO nova.compute.claims [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Claim successful on node user
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.443s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:13:09 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:13:09 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Creating image(s)
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "/opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "/opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "/opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.policy [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b2facbf385e8402c86706cc28d4707af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1318728a20064d49b9c1b2f89175a364', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Skipping network cache update for instance because it is Building. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9805}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk 1073741824" returned: 0 in 0.059s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.193s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Cannot resize image /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.objects.instance [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'migration_context' on Instance uuid d309dc50-d3b0-47a4-b263-a67ae96e2f5b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Ensure instance console log exists: /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Successfully created port: 2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [{"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:13:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Successfully updated port: 2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquired lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-changed-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Refreshing instance network info cache due to event network-changed-2749fb25-d2ff-43bb-b58c-f1d7694570de. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] Acquiring lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Updating instance_info_cache with network_info: [{"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Releasing lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Instance network_info: |[{"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] Acquired lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Refreshing network info cache for port 2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Start _get_guest_xml network_info=[{"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:13:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2095269608',display_name='tempest-VolumesActionsTest-instance-2095269608',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2095269608',id=32,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-cdvmvmi5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:10Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=d309dc50-d3b0-47a4-b263-a67ae96e2f5b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.objects.instance [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'pci_devices' on Instance uuid d309dc50-d3b0-47a4-b263-a67ae96e2f5b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:13:11 user nova-compute[71323]:   <uuid>d309dc50-d3b0-47a4-b263-a67ae96e2f5b</uuid>
Apr 14 03:13:11 user nova-compute[71323]:   <name>instance-00000020</name>
Apr 14 03:13:11 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:13:11 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:13:11 user nova-compute[71323]:   <metadata>
Apr 14 03:13:11 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:13:11 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:name>tempest-VolumesActionsTest-instance-2095269608</nova:name>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:13:11</nova:creationTime>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:13:11 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:13:11 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:owner>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:user uuid="b2facbf385e8402c86706cc28d4707af">tempest-VolumesActionsTest-121883605-project-member</nova:user>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:project uuid="1318728a20064d49b9c1b2f89175a364">tempest-VolumesActionsTest-121883605</nova:project>
Apr 14 03:13:11 user nova-compute[71323]:       </nova:owner>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:13:11 user nova-compute[71323]:       <nova:ports>
Apr 14 03:13:11 user nova-compute[71323]:         <nova:port uuid="2749fb25-d2ff-43bb-b58c-f1d7694570de">
Apr 14 03:13:11 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:13:11 user nova-compute[71323]:         </nova:port>
Apr 14 03:13:11 user nova-compute[71323]:       </nova:ports>
Apr 14 03:13:11 user nova-compute[71323]:     </nova:instance>
Apr 14 03:13:11 user nova-compute[71323]:   </metadata>
Apr 14 03:13:11 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:13:11 user nova-compute[71323]:     <system>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="serial">d309dc50-d3b0-47a4-b263-a67ae96e2f5b</entry>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="uuid">d309dc50-d3b0-47a4-b263-a67ae96e2f5b</entry>
Apr 14 03:13:11 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:13:11 user nova-compute[71323]:     </system>
Apr 14 03:13:11 user nova-compute[71323]:   </sysinfo>
Apr 14 03:13:11 user nova-compute[71323]:   <os>
Apr 14 03:13:11 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:13:11 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:13:11 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:13:11 user nova-compute[71323]:   </os>
Apr 14 03:13:11 user nova-compute[71323]:   <features>
Apr 14 03:13:11 user nova-compute[71323]:     <acpi/>
Apr 14 03:13:11 user nova-compute[71323]:     <apic/>
Apr 14 03:13:11 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:13:11 user nova-compute[71323]:   </features>
Apr 14 03:13:11 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:13:11 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:13:11 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:13:11 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:13:11 user nova-compute[71323]:   </clock>
Apr 14 03:13:11 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:13:11 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:13:11 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:13:11 user nova-compute[71323]:   </cpu>
Apr 14 03:13:11 user nova-compute[71323]:   <devices>
Apr 14 03:13:11 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:13:11 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:13:11 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk"/>
Apr 14 03:13:11 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:13:11 user nova-compute[71323]:     </disk>
Apr 14 03:13:11 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:13:11 user nova-compute[71323]:       <mac address="fa:16:3e:6c:a1:1d"/>
Apr 14 03:13:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:11 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:13:11 user nova-compute[71323]:       <target dev="tap2749fb25-d2"/>
Apr 14 03:13:11 user nova-compute[71323]:     </interface>
Apr 14 03:13:11 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:13:11 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/console.log" append="off"/>
Apr 14 03:13:11 user nova-compute[71323]:     </serial>
Apr 14 03:13:11 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:13:11 user nova-compute[71323]:     <video>
Apr 14 03:13:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:11 user nova-compute[71323]:     </video>
Apr 14 03:13:11 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:13:11 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:13:11 user nova-compute[71323]:     </rng>
Apr 14 03:13:11 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:13:11 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:13:11 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:13:11 user nova-compute[71323]:     </memballoon>
Apr 14 03:13:11 user nova-compute[71323]:   </devices>
Apr 14 03:13:11 user nova-compute[71323]: </domain>
Apr 14 03:13:11 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2095269608',display_name='tempest-VolumesActionsTest-instance-2095269608',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2095269608',id=32,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-cdvmvmi5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:10Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=d309dc50-d3b0-47a4-b263-a67ae96e2f5b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG os_vif [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2749fb25-d2, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap2749fb25-d2, col_values=(('external_ids', {'iface-id': '2749fb25-d2ff-43bb-b58c-f1d7694570de', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6c:a1:1d', 'vm-uuid': 'd309dc50-d3b0-47a4-b263-a67ae96e2f5b'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:11 user nova-compute[71323]: INFO os_vif [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2')
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:13:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] No VIF found with MAC fa:16:3e:6c:a1:1d, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:13:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Updated VIF entry in instance network info cache for port 2749fb25-d2ff-43bb-b58c-f1d7694570de. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:13:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Updating instance_info_cache with network_info: [{"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-97408815-25c0-4647-9705-d48c6a9dba20 req-86325cd9-7c96-4128-b0a4-dfef961be204 service nova] Releasing lock "refresh_cache-d309dc50-d3b0-47a4-b263-a67ae96e2f5b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] No waiting events found dispatching network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:13 user nova-compute[71323]: WARNING nova.compute.manager [req-34140f78-79b6-44ee-9fa9-b5f7d0ac91d8 req-9ef34414-e8f4-4696-b5a9-83bc8ea46eb7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received unexpected event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de for instance with vm_state building and task_state spawning.
Apr 14 03:13:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441995.399959, d309dc50-d3b0-47a4-b263-a67ae96e2f5b => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] VM Resumed (Lifecycle Event)
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Instance spawned successfully.
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681441995.40367, d309dc50-d3b0-47a4-b263-a67ae96e2f5b => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] VM Started (Lifecycle Event)
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Took 5.90 seconds to spawn the instance on the hypervisor.
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:15 user nova-compute[71323]: INFO nova.compute.manager [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Took 6.81 seconds to build instance.
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] No waiting events found dispatching network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:15 user nova-compute[71323]: WARNING nova.compute.manager [req-59ebda58-1f07-4858-9cb0-083110be1475 req-b01b7ed0-497d-401b-9180-39ee44e6d2c7 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received unexpected event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de for instance with vm_state active and task_state None.
Apr 14 03:13:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-075d00e0-e8cf-47ab-aa72-d6b4f279392c tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.911s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:20 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681441985.4405107, 4e47adee-bd94-4655-9bbd-ed59c5478531 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:20 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] VM Stopped (Lifecycle Event)
Apr 14 03:13:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ce0f5fa1-df32-4384-95be-abb2f4631f00 None None] [instance: 4e47adee-bd94-4655-9bbd-ed59c5478531] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:13:32 user nova-compute[71323]: INFO nova.compute.claims [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Claim successful on node user
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.535s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:13:32 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:13:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.policy [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd666936eb0ee43a2ad9e17a12e6e273d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd2c5b2aafaf4ca7bf62e988ac726658', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:13:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Creating image(s)
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "/opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.164s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.210s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk 1073741824" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.269s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Checking if we can resize image /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Cannot resize image /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'migration_context' on Instance uuid bd5b587d-58a9-411b-a755-67adaa6bd6c9 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Ensure instance console log exists: /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Successfully created port: 52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Successfully updated port: 52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-changed-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Refreshing instance network info cache due to event network-changed-52fd899f-496c-41d3-a9e0-6574a27e3b1b. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] Acquiring lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:34 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.neutron [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updating instance_info_cache with network_info: [{"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Instance network_info: |[{"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] Acquired lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Refreshing network info cache for port 52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Start _get_guest_xml network_info=[{"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:13:35 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:35 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-289155264',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-289155264',id=33,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-9yzst8e4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:33Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=bd5b587d-58a9-411b-a755-67adaa6bd6c9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid bd5b587d-58a9-411b-a755-67adaa6bd6c9 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:13:35 user nova-compute[71323]:   <uuid>bd5b587d-58a9-411b-a755-67adaa6bd6c9</uuid>
Apr 14 03:13:35 user nova-compute[71323]:   <name>instance-00000021</name>
Apr 14 03:13:35 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:13:35 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:13:35 user nova-compute[71323]:   <metadata>
Apr 14 03:13:35 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:13:35 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-289155264</nova:name>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:13:35</nova:creationTime>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:13:35 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:13:35 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:owner>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:13:35 user nova-compute[71323]:       </nova:owner>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:13:35 user nova-compute[71323]:       <nova:ports>
Apr 14 03:13:35 user nova-compute[71323]:         <nova:port uuid="52fd899f-496c-41d3-a9e0-6574a27e3b1b">
Apr 14 03:13:35 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Apr 14 03:13:35 user nova-compute[71323]:         </nova:port>
Apr 14 03:13:35 user nova-compute[71323]:       </nova:ports>
Apr 14 03:13:35 user nova-compute[71323]:     </nova:instance>
Apr 14 03:13:35 user nova-compute[71323]:   </metadata>
Apr 14 03:13:35 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:13:35 user nova-compute[71323]:     <system>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="serial">bd5b587d-58a9-411b-a755-67adaa6bd6c9</entry>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="uuid">bd5b587d-58a9-411b-a755-67adaa6bd6c9</entry>
Apr 14 03:13:35 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:13:35 user nova-compute[71323]:     </system>
Apr 14 03:13:35 user nova-compute[71323]:   </sysinfo>
Apr 14 03:13:35 user nova-compute[71323]:   <os>
Apr 14 03:13:35 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:13:35 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:13:35 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:13:35 user nova-compute[71323]:   </os>
Apr 14 03:13:35 user nova-compute[71323]:   <features>
Apr 14 03:13:35 user nova-compute[71323]:     <acpi/>
Apr 14 03:13:35 user nova-compute[71323]:     <apic/>
Apr 14 03:13:35 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:13:35 user nova-compute[71323]:   </features>
Apr 14 03:13:35 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:13:35 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:13:35 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:13:35 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:13:35 user nova-compute[71323]:   </clock>
Apr 14 03:13:35 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:13:35 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:13:35 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:13:35 user nova-compute[71323]:   </cpu>
Apr 14 03:13:35 user nova-compute[71323]:   <devices>
Apr 14 03:13:35 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:13:35 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:13:35 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk"/>
Apr 14 03:13:35 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:13:35 user nova-compute[71323]:     </disk>
Apr 14 03:13:35 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:13:35 user nova-compute[71323]:       <mac address="fa:16:3e:48:d8:5c"/>
Apr 14 03:13:35 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:35 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:13:35 user nova-compute[71323]:       <target dev="tap52fd899f-49"/>
Apr 14 03:13:35 user nova-compute[71323]:     </interface>
Apr 14 03:13:35 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:13:35 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/console.log" append="off"/>
Apr 14 03:13:35 user nova-compute[71323]:     </serial>
Apr 14 03:13:35 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:13:35 user nova-compute[71323]:     <video>
Apr 14 03:13:35 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:35 user nova-compute[71323]:     </video>
Apr 14 03:13:35 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:13:35 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:13:35 user nova-compute[71323]:     </rng>
Apr 14 03:13:35 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:13:35 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:13:35 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:13:35 user nova-compute[71323]:     </memballoon>
Apr 14 03:13:35 user nova-compute[71323]:   </devices>
Apr 14 03:13:35 user nova-compute[71323]: </domain>
Apr 14 03:13:35 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-289155264',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-289155264',id=33,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-9yzst8e4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:33Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=bd5b587d-58a9-411b-a755-67adaa6bd6c9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG os_vif [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52fd899f-49, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap52fd899f-49, col_values=(('external_ids', {'iface-id': '52fd899f-496c-41d3-a9e0-6574a27e3b1b', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:48:d8:5c', 'vm-uuid': 'bd5b587d-58a9-411b-a755-67adaa6bd6c9'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:35 user nova-compute[71323]: INFO os_vif [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49')
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:48:d8:5c, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updated VIF entry in instance network info cache for port 52fd899f-496c-41d3-a9e0-6574a27e3b1b. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updating instance_info_cache with network_info: [{"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e810eb6-7124-4747-a85b-133dac0cd960 req-34f08e81-418a-41e0-b004-72933fdca25e service nova] Releasing lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:37 user nova-compute[71323]: WARNING nova.compute.manager [req-a4f8657a-ab2e-47d1-8636-a65517a84477 req-b87553f5-6237-4c87-84f4-0dd8034c7b47 service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state building and task_state spawning.
Apr 14 03:13:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:38 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442018.972574, bd5b587d-58a9-411b-a755-67adaa6bd6c9 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:38 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] VM Resumed (Lifecycle Event)
Apr 14 03:13:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:13:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:13:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Instance spawned successfully.
Apr 14 03:13:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:13:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442018.9734318, bd5b587d-58a9-411b-a755-67adaa6bd6c9 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] VM Started (Lifecycle Event)
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:39 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:39 user nova-compute[71323]: INFO nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Took 6.02 seconds to spawn the instance on the hypervisor.
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:39 user nova-compute[71323]: INFO nova.compute.manager [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Took 6.94 seconds to build instance.
Apr 14 03:13:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-21da6087-3618-494b-855c-bf9621f32d9b tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.068s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:39 user nova-compute[71323]: WARNING nova.compute.manager [req-9c1c6452-73d2-49f6-ac41-9ea9fd8c6ee9 req-d3391f2f-11ff-4892-ac90-a7082e50bbea service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state active and task_state None.
Apr 14 03:13:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:40 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:13:40 user nova-compute[71323]: INFO nova.compute.claims [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Claim successful on node user
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.423s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:13:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.policy [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ba8614e27c65428d92e4bf01969976e7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0096d0892acf4d5f8a0fa290a18b451a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:13:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Creating image(s)
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "/opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "/opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "/opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Ignoring supplied device name: /dev/vdb
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.061s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.156s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk 1073741824" returned: 0 in 0.054s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.217s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:41 user nova-compute[71323]: INFO nova.compute.manager [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Attaching volume eb69c0bd-af4f-438c-b96a-94c59261813a to /dev/vdb
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:41 user nova-compute[71323]: DEBUG os_brick.utils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[609f20d8-e942-48a3-bb0d-e7b8b24caa0c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ed3dc78-5e76-4de8-8ddc-50720184dcd7]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c00d7e9c-f25b-482b-9ac1-b82602f9e1c9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c4883ae-965e-44d5-ac1f-16d38602d6c2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Cannot resize image /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'migration_context' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:13:42 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c49d98f8-1786-45eb-a71b-1e815b88c283]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG os_brick.utils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== get_connector_properties: return (139ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Updating existing volume attachment record: 84b39d34-19fb-4a0d-98ba-939032766d98 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Ensure instance console log exists: /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Successfully created port: 73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Successfully updated port: 73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquired lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing instance network info cache due to event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:13:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] Acquiring lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating instance_info_cache with network_info: [{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Releasing lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Instance network_info: |[{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] Acquired lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Start _get_guest_xml network_info=[{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:13:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:44 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-79587855',display_name='tempest-SnapshotDataIntegrityTests-server-79587855',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-79587855',id=34,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN4ITzOqj8xEk6dTBZ8Ot32M8Esvy3C6MqRifOQoX4ksa7bmDWj6LGvCYEbIDUI0lXUxkiCpG51Z30GVyclYEqcHtnzy83lAYeIu0VXyXlLqttu//3qnh/G2175vPwLjXQ==',key_name='tempest-SnapshotDataIntegrityTests-1339724753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0096d0892acf4d5f8a0fa290a18b451a',ramdisk_id='',reservation_id='r-pmhvgxq9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-450991924',owner_user_name='tempest-SnapshotDataIntegrityTests-450991924-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:41Z,user_data=None,user_id='ba8614e27c65428d92e4bf01969976e7',uuid=9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converting VIF {"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'pci_devices' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:13:44 user nova-compute[71323]:   <uuid>9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4</uuid>
Apr 14 03:13:44 user nova-compute[71323]:   <name>instance-00000022</name>
Apr 14 03:13:44 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:13:44 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:13:44 user nova-compute[71323]:   <metadata>
Apr 14 03:13:44 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:13:44 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-79587855</nova:name>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:13:44</nova:creationTime>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:13:44 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:13:44 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:owner>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:user uuid="ba8614e27c65428d92e4bf01969976e7">tempest-SnapshotDataIntegrityTests-450991924-project-member</nova:user>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:project uuid="0096d0892acf4d5f8a0fa290a18b451a">tempest-SnapshotDataIntegrityTests-450991924</nova:project>
Apr 14 03:13:44 user nova-compute[71323]:       </nova:owner>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:13:44 user nova-compute[71323]:       <nova:ports>
Apr 14 03:13:44 user nova-compute[71323]:         <nova:port uuid="73d42070-416a-497f-b3d4-8467d8255c5f">
Apr 14 03:13:44 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:13:44 user nova-compute[71323]:         </nova:port>
Apr 14 03:13:44 user nova-compute[71323]:       </nova:ports>
Apr 14 03:13:44 user nova-compute[71323]:     </nova:instance>
Apr 14 03:13:44 user nova-compute[71323]:   </metadata>
Apr 14 03:13:44 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:13:44 user nova-compute[71323]:     <system>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="serial">9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4</entry>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="uuid">9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4</entry>
Apr 14 03:13:44 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:13:44 user nova-compute[71323]:     </system>
Apr 14 03:13:44 user nova-compute[71323]:   </sysinfo>
Apr 14 03:13:44 user nova-compute[71323]:   <os>
Apr 14 03:13:44 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:13:44 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:13:44 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:13:44 user nova-compute[71323]:   </os>
Apr 14 03:13:44 user nova-compute[71323]:   <features>
Apr 14 03:13:44 user nova-compute[71323]:     <acpi/>
Apr 14 03:13:44 user nova-compute[71323]:     <apic/>
Apr 14 03:13:44 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:13:44 user nova-compute[71323]:   </features>
Apr 14 03:13:44 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:13:44 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:13:44 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:13:44 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:13:44 user nova-compute[71323]:   </clock>
Apr 14 03:13:44 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:13:44 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:13:44 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:13:44 user nova-compute[71323]:   </cpu>
Apr 14 03:13:44 user nova-compute[71323]:   <devices>
Apr 14 03:13:44 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:13:44 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:13:44 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk"/>
Apr 14 03:13:44 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:13:44 user nova-compute[71323]:     </disk>
Apr 14 03:13:44 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:13:44 user nova-compute[71323]:       <mac address="fa:16:3e:dd:ec:3f"/>
Apr 14 03:13:44 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:44 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:13:44 user nova-compute[71323]:       <target dev="tap73d42070-41"/>
Apr 14 03:13:44 user nova-compute[71323]:     </interface>
Apr 14 03:13:44 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:13:44 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/console.log" append="off"/>
Apr 14 03:13:44 user nova-compute[71323]:     </serial>
Apr 14 03:13:44 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:13:44 user nova-compute[71323]:     <video>
Apr 14 03:13:44 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:13:44 user nova-compute[71323]:     </video>
Apr 14 03:13:44 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:13:44 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:13:44 user nova-compute[71323]:     </rng>
Apr 14 03:13:44 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:13:44 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:13:44 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:13:44 user nova-compute[71323]:     </memballoon>
Apr 14 03:13:44 user nova-compute[71323]:   </devices>
Apr 14 03:13:44 user nova-compute[71323]: </domain>
Apr 14 03:13:44 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-79587855',display_name='tempest-SnapshotDataIntegrityTests-server-79587855',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-79587855',id=34,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN4ITzOqj8xEk6dTBZ8Ot32M8Esvy3C6MqRifOQoX4ksa7bmDWj6LGvCYEbIDUI0lXUxkiCpG51Z30GVyclYEqcHtnzy83lAYeIu0VXyXlLqttu//3qnh/G2175vPwLjXQ==',key_name='tempest-SnapshotDataIntegrityTests-1339724753',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0096d0892acf4d5f8a0fa290a18b451a',ramdisk_id='',reservation_id='r-pmhvgxq9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-450991924',owner_user_name='tempest-SnapshotDataIntegrityTests-450991924-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:13:41Z,user_data=None,user_id='ba8614e27c65428d92e4bf01969976e7',uuid=9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converting VIF {"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG os_vif [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap73d42070-41, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap73d42070-41, col_values=(('external_ids', {'iface-id': '73d42070-416a-497f-b3d4-8467d8255c5f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:ec:3f', 'vm-uuid': '9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:44 user nova-compute[71323]: INFO os_vif [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41')
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No VIF found with MAC fa:16:3e:dd:ec:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updated VIF entry in instance network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating instance_info_cache with network_info: [{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:13:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-08135dc2-7657-4c23-b082-a9ad9be9230c req-f6f62170-8ecd-45cd-8835-b0a783c61469 service nova] Releasing lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] No waiting events found dispatching network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:45 user nova-compute[71323]: WARNING nova.compute.manager [req-cb3e9f21-3785-41f7-ba41-576b660839a7 req-84b3fc97-315a-48af-bb2f-570999d3caa7 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received unexpected event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f for instance with vm_state building and task_state spawning.
Apr 14 03:13:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pjXgqBiH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:13:46 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:13:46 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:13:46 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb5a2908-1fd4-4334-8387-07b39c4c0a0f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:13:46 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[15b6d040-b02b-4a60-ba35-769a515b0b73]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a] added
Apr 14 03:13:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d33d51c0-f63d-4ae6-97f2-91f55ee55701]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2850c3b3-f45c-429d-9cdc-6308d8b1b7a3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pjXgqBiH {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pjXgqBiH" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e68130b-1252-4ebb-899e-f94502d4d5cf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pjXgqBiH'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[28793f45-2fac-4f1f-bd51-36c282d55b1a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ad6fb1a-a1cf-4361-9288-6864d8dc8370]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93b8a745-98dc-4b2a-bebd-1e2d87b46054]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260]
Apr 14 03:13:47 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260] successful.
Apr 14 03:13:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[da0f92ee-bc2b-4115-95e7-06e51235bba8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8045c613-3b6f-4913-8624-716f925c02fb]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:47 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:47 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.315s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] HCTL ('35', '-', '-', 0) found on session 35 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ffd9cd4-16a2-47f1-8202-f24ff889e558]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Searching for a device in session 35 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pjXgqBiH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442027.8184597, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] VM Resumed (Lifecycle Event)
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Instance spawned successfully.
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442027.8203526, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] VM Started (Lifecycle Event)
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:13:47 user nova-compute[71323]: INFO nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Took 6.51 seconds to spawn the instance on the hypervisor.
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] No waiting events found dispatching network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:13:47 user nova-compute[71323]: WARNING nova.compute.manager [req-eeeeebdf-a984-4f01-bd39-df8402c21c0d req-6f0676b9-520e-4f04-b4dc-0271009b83df service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received unexpected event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f for instance with vm_state building and task_state spawning.
Apr 14 03:13:48 user nova-compute[71323]: INFO nova.compute.manager [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Took 7.27 seconds to build instance.
Apr 14 03:13:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fbe18a6f-9a78-47ad-bf07-eb632b12d300 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.382s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.356s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== connect_volume: return (1358ms) {'type': 'block', 'scsi_wwn': '26130316163326664', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26130316163326664', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:13:48 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:13:48 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:13:48 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:13:48 user nova-compute[71323]:   <serial>eb69c0bd-af4f-438c-b96a-94c59261813a</serial>
Apr 14 03:13:48 user nova-compute[71323]: </disk>
Apr 14 03:13:48 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] No VIF found with MAC fa:16:3e:90:9c:13, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:13:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e7ead739-5a9a-4e01-ae56-98a99efbb448 tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.958s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'flavor' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.053s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:53 user nova-compute[71323]: INFO nova.compute.manager [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attaching volume 7fff5d89-4516-4be1-9698-f088498889ff to /dev/vdb
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.utils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f9e47e0-d5dc-4fe4-a15f-7003ee9f403e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[46ba48eb-95c4-43cf-8820-d00fc4ccbeda]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[78d6995d-c9bd-47e2-ac43-c998911c9961]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efe79d2b-bff1-4799-971c-8ebb5ce1052e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:13:53 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:13:53 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9388cb93-f670-4fe7-87ff-af7a07c67a0f]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG os_brick.utils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:13:53 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating existing volume attachment record: 6bc4d910-81cd-4cad-b971-4d1cd3ba9cf2 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:13:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 7f542934-641f-405f-811c-3601a3be389b {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 24d1359d-2dcd-46d7-90f3-3801df9be357 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 5d0f51bd-b28d-4db1-be36-cec6cc92231e {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid d309dc50-d3b0-47a4-b263-a67ae96e2f5b {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid bd5b587d-58a9-411b-a755-67adaa6bd6c9 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "7f542934-641f-405f-811c-3601a3be389b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.066s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.079s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "7f542934-641f-405f-811c-3601a3be389b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.082s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.090s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.098s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.108s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.100s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.108s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.106s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:13:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:13:58 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:13:58 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:13:58 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260" returned: 21 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a2ddadd-1edd-43ae-a19b-e2ef3dccfee0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:13:58 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[30acab0c-5bdf-4a8e-86b3-f43345f89dd4]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff] added
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93eea5ac-cd05-4de0-98cf-40ab819724f2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[063d412f-20e5-475c-a820-54cc0f7d465b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v weNMmovz {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v weNMmovz" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2286f183-f8bf-4cd0-bf83-645634ee9215]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'weNMmovz'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e622439-6049-44b7-9ae9-0a3449fa7ca4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2be3f33e-d70f-40ff-814c-776c35e42135]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --login" returned: 0 in 0.085s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b132447c-921a-4c1e-8cd4-42b0694285b8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260]
Apr 14 03:13:58 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260] successful.
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a51d784d-1400-43e6-b013-06811f615861]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3f7a6c8-93e9-47e1-a8dc-55999433268f]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:13:58 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:13:58 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.379s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] HCTL ('36', '-', '-', 0) found on session 36 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[71e312a4-fe6e-4896-a92c-5ec6559061a7]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Searching for a device in session 36 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:13:58 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.421s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] <== connect_volume: return (1422ms) {'type': 'block', 'scsi_wwn': '23062626637306135', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062626637306135', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'flavor' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:13:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:13:59 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:13:59 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:13:59 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:13:59 user nova-compute[71323]:   <serial>7fff5d89-4516-4be1-9698-f088498889ff</serial>
Apr 14 03:13:59 user nova-compute[71323]: </disk>
Apr 14 03:13:59 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No VIF found with MAC fa:16:3e:af:d1:f9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6f19d01f-dddd-4d94-91e5-83ef56bcf2cf tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.860s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 4.549s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.037s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:01 user nova-compute[71323]: INFO nova.compute.manager [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Rescuing
Apr 14 03:14:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:14:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG nova.compute.manager [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG nova.compute.manager [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:02 user nova-compute[71323]: WARNING nova.compute.manager [req-e192af85-fe42-4a18-ae5f-9a4d69bf13c0 req-600d6c06-9cba-4e1e-ae76-2a16a6879147 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state active and task_state rescuing.
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:02 user nova-compute[71323]: INFO nova.compute.manager [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Terminating instance
Apr 14 03:14:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance destroyed successfully.
Apr 14 03:14:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attempting a stable device rescue
Apr 14 03:14:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:14:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Creating image(s)
Apr 14 03:14:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'trusted_certs' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "e823b9712be850c2c32801ce101a377eb1abc784" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "e823b9712be850c2c32801ce101a377eb1abc784" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:14:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Instance destroyed successfully.
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid ac032176-63bc-457c-ad53-5b4a854eec34 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-1086696024',display_name='tempest-ServerRescueNegativeTestJSON-server-1086696024',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-1086696024',id=4,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T02:59:56Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-d1yclxnx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T02:59:57Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=ac032176-63bc-457c-ad53-5b4a854eec34,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='rescued') vif={"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "59fef74f-8c76-4dd5-bc99-311eea495950", "address": "fa:16:3e:b2:39:e3", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap59fef74f-8c", "ovs_interfaceid": "59fef74f-8c76-4dd5-bc99-311eea495950", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG os_vif [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap59fef74f-8c, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:04 user nova-compute[71323]: INFO os_vif [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b2:39:e3,bridge_name='br-int',has_traffic_filtering=True,id=59fef74f-8c76-4dd5-bc99-311eea495950,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap59fef74f-8c')
Apr 14 03:14:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Deleting instance files /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34_del
Apr 14 03:14:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Deletion of /opt/stack/data/nova/instances/ac032176-63bc-457c-ad53-5b4a854eec34_del complete
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.part --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.virt.images [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] b96871c7-2eee-43a3-82b5-3a0299fdd7a8 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.part /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:04 user nova-compute[71323]: INFO nova.compute.manager [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Took 1.47 seconds to destroy the instance on the hypervisor.
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:04 user nova-compute[71323]: WARNING nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state active and task_state rescuing.
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-unplugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Acquiring lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] Lock "ac032176-63bc-457c-ad53-5b4a854eec34-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] No waiting events found dispatching network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:04 user nova-compute[71323]: WARNING nova.compute.manager [req-248e3821-da12-4792-8662-f6d5fd95ae0a req-719c32d4-000a-4553-a445-e5885642d472 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received unexpected event network-vif-plugged-59fef74f-8c76-4dd5-bc99-311eea495950 for instance with vm_state rescued and task_state deleting.
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.part /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.converted" returned: 0 in 0.390s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784.converted --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "e823b9712be850c2c32801ce101a377eb1abc784" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.646s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "e823b9712be850c2c32801ce101a377eb1abc784" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "e823b9712be850c2c32801ce101a377eb1abc784" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:04 user nova-compute[71323]: INFO nova.compute.manager [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Detaching volume eb69c0bd-af4f-438c-b96a-94c59261813a
Apr 14 03:14:04 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Took 0.60 seconds to deallocate network for instance.
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.virt.block_device [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Attempting to driver detach volume eb69c0bd-af4f-438c-b96a-94c59261813a from mountpoint /dev/vdb
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Attempting to detach device vdb from instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:14:05 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:14:05 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:14:05 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:   <serial>eb69c0bd-af4f-438c-b96a-94c59261813a</serial>
Apr 14 03:14:05 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:14:05 user nova-compute[71323]: </disk>
Apr 14 03:14:05 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784 --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784,backing_fmt=raw /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully detached device vdb from instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 from the persistent domain config.
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:14:05 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:14:05 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:14:05 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:   <serial>eb69c0bd-af4f-438c-b96a-94c59261813a</serial>
Apr 14 03:14:05 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:14:05 user nova-compute[71323]: </disk>
Apr 14 03:14:05 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784,backing_fmt=raw /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "e823b9712be850c2c32801ce101a377eb1abc784" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.203s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'migration_context' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start _get_guest_xml network_info=[{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "vif_mac": "fa:16:3e:af:d1:f9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'b96871c7-2eee-43a3-82b5-3a0299fdd7a8', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '6bc4d910-81cd-4cad-b971-4d1cd3ba9cf2', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, 'status': 'reserved', 'instance': '8a67985b-1938-4b7d-b486-13f6811611f3', 'attached_at': '', 'detached_at': '', 'volume_id': '7fff5d89-4516-4be1-9698-f088498889ff', 'serial': '7fff5d89-4516-4be1-9698-f088498889ff'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'resources' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'numa_topology' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:05 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'vcpu_model' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:05 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[51d956bc-02ca-4ae8-8602-04e1737e54b1]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = weNMmovz\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 03:14:05 user nova-compute[71323]: node.name = iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:14:05 user nova-compute[71323]: node.tpgt = -1
Apr 14 03:14:05 user nova-compute[71323]: node.startup = automatic
Apr 14 03:14:05 user nova-compute[71323]: node.leading_login = No
Apr 14 03:14:05 user nova-compute[71323]: iface.iscsi_ifacename = default
Apr 14 03:14:05 user nova-compute[71323]: iface.net_ifacename = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.ipaddress = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.prefix_len = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.hwaddress = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.transport_name = tcp
Apr 14 03:14:05 user nova-compute[71323]: iface.initiatorname = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.vlan_id = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.vlan_priority = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.vlan_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.iface_num = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.mtu = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.port = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.bootproto = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.subnet_mask = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.gateway = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_alt_client_id = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_dns = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_learn_iqn = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_vendor_id_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_vendor_id = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.dhcp_slp_da = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.fragmentation = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.gratuitous_arp = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.incoming_forwarding = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.tos_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.tos = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.ttl = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.delayed_ack = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.tcp_nagle = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.tcp_wsf_state = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.tcp_wsf = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.tcp_timer_scale = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.tcp_timestamp = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.redirect = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.def_task_mgmt_timeout = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.header_digest = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.data_digest = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.immediate_data = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.initial_r2t = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.data_seq_inorder = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.data_pdu_inorder = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.erl = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.max_receive_data_len = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.first_burst_len = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.max_outstanding_r2t = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.max_burst_len = 0
Apr 14 03:14:05 user nova-compute[71323]: iface.chap_auth = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.bidi_chap = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.strict_login_compliance = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.discovery_auth = <empty>
Apr 14 03:14:05 user nova-compute[71323]: iface.discovery_logout = <empty>
Apr 14 03:14:05 user nova-compute[71323]: node.discovery_address = <empty>
Apr 14 03:14:05 user nova-compute[71323]: node.discovery_port = 0
Apr 14 03:14:05 user nova-compute[71323]: node.discovery_type = static
Apr 14 03:14:05 user nova-compute[71323]: node.session.initial_cmdsn = 0
Apr 14 03:14:05 user nova-compute[71323]: node.session.initial_login_retry_max = 8
Apr 14 03:14:05 user nova-compute[71323]: node.session.xmit_thread_priority = -20
Apr 14 03:14:05 user nova-compute[71323]: node.session.cmds_max = 128
Apr 14 03:14:05 user nova-compute[71323]: node.session.queue_depth = 32
Apr 14 03:14:05 user nova-compute[71323]: node.session.nr_sessions = 1
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.authmethod = CHAP
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.username = weNMmovz
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.password = ***
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.username_in = <empty>
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.password_in = <empty>
Apr 14 03:14:05 user nova-compute[71323]: node.session.auth.chap_algs = MD5
Apr 14 03:14:05 user nova-compute[71323]: node.session.timeo.replacement_timeout = 120
Apr 14 03:14:05 user nova-compute[71323]: node.session.err_timeo.abort_timeout = 15
Apr 14 03:14:05 user nova-compute[71323]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 03:14:05 user nova-compute[71323]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 03:14:05 user nova-compute[71323]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.FastAbort = Yes
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.InitialR2T = No
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.ImmediateData = Yes
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.MaxConnections = 1
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 03:14:05 user nova-compute[71323]: node.session.iscsi.ERL = 0
Apr 14 03:14:05 user nova-compute[71323]: node.session.scan = manual
Apr 14 03:14:05 user nova-compute[71323]: node.session.reopen_max = 0
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].address = 172.16.0.220
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].port = 3260
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].startup = manual
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].tcp.window_size = 524288
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].tcp.type_of_service = 0
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].timeo.logout_timeout = 15
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].timeo.login_timeout = 15
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].timeo.auth_timeout = 45
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.DataDigest = None
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.IFMarker = No
Apr 14 03:14:05 user nova-compute[71323]: node.conn[0].iscsi.OFMarker = No
Apr 14 03:14:05 user nova-compute[71323]: # END RECORD
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ca142a6-01c3-4fa2-b963-7b659cca3899]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a22878d-3fdf-4de4-a60c-9239635fee28]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v weNMmovz {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.username -v weNMmovz" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f6c1f1f-d73c-4341-b64c-8837ece2d33b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'weNMmovz'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fab6e71-38e5-480d-8dd9-dd423d985175]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdbb9395-4c73-4260-b6c8-d5ff3954e94b]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.173s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] HCTL ('36', '0', '0', 0) found on session 36 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Scanning host 36 c: 0, t: 0, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[defc6b80-3c67-4c3d-8fec-aa671076e096]: (4, ('0 0 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Searching for a device in session 36 and hctl ('36', '0', '0', 0) yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.198s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] <== connect_volume: return (198ms) {'type': 'block', 'scsi_wwn': '23062626637306135', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23062626637306135', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-196777223',display_name='tempest-ServerStableDeviceRescueTest-server-196777223',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-196777223',id=26,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7w1V2h51K5WeC0lNrY7+q4q2XqyPZ1TB9opotq+DNhc9r2qQzgCUPc3xsU2/ON8yD/igMXO0IAbOHg83C68rdtapd1/KHmCO4Nj4cZtEPuSMjanMb8ZhuWuQX+Z2NrHw==',key_name='tempest-keypair-1447416444',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:10:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='82eb4fbf55f04294a3d97800142bdca9',ramdisk_id='',reservation_id='r-pniwrgja',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1642713032',owner_user_name='tempest-ServerStableDeviceRescueTest-1642713032-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:12:05Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3c683fe5636b49858857033797ff8e68',uuid=8a67985b-1938-4b7d-b486-13f6811611f3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "vif_mac": "fa:16:3e:af:d1:f9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converting VIF {"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "vif_mac": "fa:16:3e:af:d1:f9"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'pci_devices' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:14:05 user nova-compute[71323]:   <uuid>8a67985b-1938-4b7d-b486-13f6811611f3</uuid>
Apr 14 03:14:05 user nova-compute[71323]:   <name>instance-0000001a</name>
Apr 14 03:14:05 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:14:05 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:14:05 user nova-compute[71323]:   <metadata>
Apr 14 03:14:05 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:14:05 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-196777223</nova:name>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:14:05</nova:creationTime>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:14:05 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:14:05 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:owner>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:user uuid="3c683fe5636b49858857033797ff8e68">tempest-ServerStableDeviceRescueTest-1642713032-project-member</nova:user>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:project uuid="82eb4fbf55f04294a3d97800142bdca9">tempest-ServerStableDeviceRescueTest-1642713032</nova:project>
Apr 14 03:14:05 user nova-compute[71323]:       </nova:owner>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:14:05 user nova-compute[71323]:       <nova:ports>
Apr 14 03:14:05 user nova-compute[71323]:         <nova:port uuid="2d081ef4-9756-4571-b28c-0d4582c58f0a">
Apr 14 03:14:05 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:14:05 user nova-compute[71323]:         </nova:port>
Apr 14 03:14:05 user nova-compute[71323]:       </nova:ports>
Apr 14 03:14:05 user nova-compute[71323]:     </nova:instance>
Apr 14 03:14:05 user nova-compute[71323]:   </metadata>
Apr 14 03:14:05 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:14:05 user nova-compute[71323]:     <system>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="serial">8a67985b-1938-4b7d-b486-13f6811611f3</entry>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="uuid">8a67985b-1938-4b7d-b486-13f6811611f3</entry>
Apr 14 03:14:05 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:14:05 user nova-compute[71323]:     </system>
Apr 14 03:14:05 user nova-compute[71323]:   </sysinfo>
Apr 14 03:14:05 user nova-compute[71323]:   <os>
Apr 14 03:14:05 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:14:05 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:14:05 user nova-compute[71323]:   </os>
Apr 14 03:14:05 user nova-compute[71323]:   <features>
Apr 14 03:14:05 user nova-compute[71323]:     <acpi/>
Apr 14 03:14:05 user nova-compute[71323]:     <apic/>
Apr 14 03:14:05 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:14:05 user nova-compute[71323]:   </features>
Apr 14 03:14:05 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:14:05 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:14:05 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:14:05 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:14:05 user nova-compute[71323]:   </clock>
Apr 14 03:14:05 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:14:05 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:14:05 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:14:05 user nova-compute[71323]:   </cpu>
Apr 14 03:14:05 user nova-compute[71323]:   <devices>
Apr 14 03:14:05 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:14:05 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:14:05 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk"/>
Apr 14 03:14:05 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:     </disk>
Apr 14 03:14:05 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:14:05 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:14:05 user nova-compute[71323]:       <source dev="/dev/sde"/>
Apr 14 03:14:05 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:       <serial>7fff5d89-4516-4be1-9698-f088498889ff</serial>
Apr 14 03:14:05 user nova-compute[71323]:     </disk>
Apr 14 03:14:05 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:14:05 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:14:05 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue"/>
Apr 14 03:14:05 user nova-compute[71323]:       <target dev="vdc" bus="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:       <boot order="1"/>
Apr 14 03:14:05 user nova-compute[71323]:     </disk>
Apr 14 03:14:05 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:14:05 user nova-compute[71323]:       <mac address="fa:16:3e:af:d1:f9"/>
Apr 14 03:14:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:14:05 user nova-compute[71323]:       <target dev="tap2d081ef4-97"/>
Apr 14 03:14:05 user nova-compute[71323]:     </interface>
Apr 14 03:14:05 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:14:05 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/console.log" append="off"/>
Apr 14 03:14:05 user nova-compute[71323]:     </serial>
Apr 14 03:14:05 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:14:05 user nova-compute[71323]:     <video>
Apr 14 03:14:05 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:14:05 user nova-compute[71323]:     </video>
Apr 14 03:14:05 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:14:05 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:14:05 user nova-compute[71323]:     </rng>
Apr 14 03:14:05 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:14:05 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:14:05 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:14:05 user nova-compute[71323]:     </memballoon>
Apr 14 03:14:05 user nova-compute[71323]:   </devices>
Apr 14 03:14:05 user nova-compute[71323]: </domain>
Apr 14 03:14:05 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance destroyed successfully.
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442045.5265186, 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully detached device vdb from instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 from the live domain config.
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pjXgqBiH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] No VIF found with MAC fa:16:3e:af:d1:f9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[252d63fa-d81a-4ef1-85cc-f5e3a002bd65]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:14:05 user nova-compute[71323]: No targets found.
Apr 14 03:14:05 user nova-compute[71323]: iSNS:
Apr 14 03:14:05 user nova-compute[71323]: No targets found.
Apr 14 03:14:05 user nova-compute[71323]: STATIC:
Apr 14 03:14:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:14:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53
Apr 14 03:14:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:14:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:14:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a
Apr 14 03:14:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:05 user nova-compute[71323]: FIRMWARE:
Apr 14 03:14:05 user nova-compute[71323]: No targets found.
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:14:05 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:14:05 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ea0fa47-ed0f-4e5a-9033-864ab0c64356]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.515s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd982597-1a6d-4cf0-9ee0-ea13d74b36f0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a (non-flash)
Apr 14 03:14:05 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd4dc0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:14:05 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Deleted allocations for instance ac032176-63bc-457c-ad53-5b4a854eec34
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[df0c9322-b365-49ea-bce4-d5825af8e077]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8631563b-0e8d-4233-90e4-b38db1513ccd]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bc9d8c3-94d2-4978-9ddc-0d7fb7414386]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d5bc8fca-df5d-4014-bd76-e7e6fd9b594c tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "ac032176-63bc-457c-ad53-5b4a854eec34" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.804s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4acbebb0-3963-4204-8d31-032c015bdf88]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6650b455-ba87-4b12-ad25-f9597cf5ec42]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a015a7a-232f-4ea9-ac12-2674f5018d97]: (4, ('Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260]\nLogout of [sid: 35, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260]
Apr 14 03:14:05 user nova-compute[71323]: Logout of [sid: 35, target: iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a, portal: 172.16.0.220,3260] successful.
Apr 14 03:14:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[acf2965d-bcdc-4fa3-9e61-aeb3713f0810]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.354s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== disconnect_volume: return (355ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-a044ccf7-3bfd-4d93-9bcc-218af9cf3089 req-868dda2f-ae86-45b5-9fbc-de01b9c69b47 service nova] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Received event network-vif-deleted-59fef74f-8c76-4dd5-bc99-311eea495950 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'flavor' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a18ebcfa-8927-4fde-9710-abe0a21f955f tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.857s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 1.722s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:06 user nova-compute[71323]: INFO nova.compute.manager [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Terminating instance
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Instance destroyed successfully.
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.objects.instance [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lazy-loading 'resources' on Instance uuid 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:11:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1353831390',display_name='tempest-VolumesSnapshotTestJSON-instance-1353831390',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1353831390',id=29,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:11:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4549f6080dcf4c739ac03ef643801fbc',ramdisk_id='',reservation_id='r-lni03qpo',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-423841644',owner_user_name='tempest-VolumesSnapshotTestJSON-423841644-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:11:58Z,user_data=None,user_id='0128a733bd6a4120addba5eda1cc05fb',uuid=0708e26d-9d20-4cb0-9b8c-76249f74f0a2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converting VIF {"id": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "address": "fa:16:3e:90:9c:13", "network": {"id": "bdc97991-26cc-4e5b-8d38-a1f750e7dd2c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-214912021-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4549f6080dcf4c739ac03ef643801fbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6f05389c-b6", "ovs_interfaceid": "6f05389c-b6d4-40ce-afa3-06e781a35ba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_vif [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6f05389c-b6, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:14:07 user nova-compute[71323]: INFO os_vif [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:90:9c:13,bridge_name='br-int',has_traffic_filtering=True,id=6f05389c-b6d4-40ce-afa3-06e781a35ba5,network=Network(bdc97991-26cc-4e5b-8d38-a1f750e7dd2c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6f05389c-b6')
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pjXgqBiH', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:07 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccedeea1-7d4b-41e4-ae74-98b344421886]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:14:07 user nova-compute[71323]: No targets found.
Apr 14 03:14:07 user nova-compute[71323]: iSNS:
Apr 14 03:14:07 user nova-compute[71323]: No targets found.
Apr 14 03:14:07 user nova-compute[71323]: STATIC:
Apr 14 03:14:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:14:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53
Apr 14 03:14:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:14:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:14:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:07 user nova-compute[71323]: FIRMWARE:
Apr 14 03:14:07 user nova-compute[71323]: No targets found.
Apr 14 03:14:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:14:07 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:14:07 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:eb69c0bd-af4f-438c-b96a-94c59261813a', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[66b90cfb-33c9-47f3-8f7c-d58229e5e704]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e923454a-e630-45a0-9b0d-bce1870b2a14]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:07 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae016c280>, {}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Disconnecting from: [] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.085s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] <== disconnect_volume: return (86ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:14:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Deleting instance files /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2_del
Apr 14 03:14:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Deletion of /opt/stack/data/nova/instances/0708e26d-9d20-4cb0-9b8c-76249f74f0a2_del complete
Apr 14 03:14:08 user nova-compute[71323]: INFO nova.compute.manager [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Took 1.26 seconds to destroy the instance on the hypervisor.
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.151s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:08 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Took 0.70 seconds to deallocate network for instance.
Apr 14 03:14:08 user nova-compute[71323]: WARNING nova.volume.cinder [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Attachment 84b39d34-19fb-4a0d-98ba-939032766d98 does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = 84b39d34-19fb-4a0d-98ba-939032766d98. (HTTP 404) (Request-ID: req-14d6005c-2c05-49f0-aaa2-45a9c22839dd)
Apr 14 03:14:08 user nova-compute[71323]: INFO nova.compute.manager [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Took 0.03 seconds to detach 1 volumes for instance.
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-unplugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] No waiting events found dispatching network-vif-unplugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-unplugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Acquiring lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] No waiting events found dispatching network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:08 user nova-compute[71323]: WARNING nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received unexpected event network-vif-plugged-6f05389c-b6d4-40ce-afa3-06e781a35ba5 for instance with vm_state active and task_state deleting.
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:08 user nova-compute[71323]: WARNING nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state active and task_state rescuing.
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:08 user nova-compute[71323]: WARNING nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state active and task_state rescuing.
Apr 14 03:14:08 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae128952-ef68-44de-93fc-7fea6cbbdbc5 req-93601d27-7021-4877-b002-8ee2ebbc5ff5 service nova] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Received event network-vif-deleted-6f05389c-b6d4-40ce-afa3-06e781a35ba5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.618s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:09 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Deleted allocations for instance 0708e26d-9d20-4cb0-9b8c-76249f74f0a2
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf912c97-3ad4-4b21-b283-539752caa9cf tempest-VolumesSnapshotTestJSON-423841644 tempest-VolumesSnapshotTestJSON-423841644-project-member] Lock "0708e26d-9d20-4cb0-9b8c-76249f74f0a2" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.851s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 8a67985b-1938-4b7d-b486-13f6811611f3 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442049.6613755, 8a67985b-1938-4b7d-b486-13f6811611f3 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:09 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Resumed (Lifecycle Event)
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b5decce8-13fc-41bb-abf5-db5abdd9baea tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442049.6629245, 8a67985b-1938-4b7d-b486-13f6811611f3 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:09 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Started (Lifecycle Event)
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7627MB free_disk=26.288070678710938GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7f542934-641f-405f-811c-3601a3be389b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 5d0f51bd-b28d-4db1-be36-cec6cc92231e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d309dc50-d3b0-47a4-b263-a67ae96e2f5b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1792MB phys_disk=40GB used_disk=7GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.453s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:14:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:14:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing instance network info cache due to event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updated VIF entry in instance network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-558f5972-ed03-4aa3-ba59-83b243971596 req-4b002fdc-1f62-4929-ad20-6e1fe7129a93 service nova] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:14:19 user nova-compute[71323]: INFO nova.compute.claims [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Claim successful on node user
Apr 14 03:14:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442043.7174594, ac032176-63bc-457c-ad53-5b4a854eec34 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:19 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] VM Stopped (Lifecycle Event)
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-85e9c443-b715-41dc-9ff8-65f12442c65c None None] [instance: ac032176-63bc-457c-ad53-5b4a854eec34] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.411s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:14:19 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:14:19 user nova-compute[71323]: INFO nova.virt.block_device [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Booting with volume 7378e388-0e78-4212-b0ce-a1e082ba31ab at /dev/vda
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.policy [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7dea23743ba945c4a9d539b8e277c653', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4075885e26cb4adaad24d3ae6217fe35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.utils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ef4ad7d-75c0-4911-8f66-f6b77b4e8642]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ab00f84-a2c9-4c0d-889e-743170802240]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c00c2604-f692-4541-b48a-85e12f010da8]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2a7099b-60d3-494e-a9c1-648c607d01d8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:14:19 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:14:19 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2bfedfd-518c-4c87-97bd-97f3d61d4930]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG os_brick.utils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:19 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating existing volume attachment record: 237abed3-9ad3-4617-8ee3-b0428e0a2a34 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:14:20 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Successfully created port: f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Successfully updated port: f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-changed-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Refreshing instance network info cache due to event network-changed-f1dbc6da-e8f9-4962-ac4c-db69105689f9. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance network_info: |[{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:21 user nova-compute[71323]: DEBUG nova.network.neutron [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Refreshing network info cache for port f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:14:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updated VIF entry in instance network info cache for port f1dbc6da-e8f9-4962-ac4c-db69105689f9. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:14:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ea5ca8ac-4152-4422-a941-788d63ebb72c req-778e0068-c5b1-4f3e-8647-03779c4ace8e service nova] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:22 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442047.7906039, 0708e26d-9d20-4cb0-9b8c-76249f74f0a2 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:22 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] VM Stopped (Lifecycle Event)
Apr 14 03:14:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a537008d-8d1d-4d55-a043-caaff6f72c1f None None] [instance: 0708e26d-9d20-4cb0-9b8c-76249f74f0a2] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:14:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Creating image(s)
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Ensure instance console log exists: /opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start _get_guest_xml network_info=[{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '237abed3-9ad3-4617-8ee3-b0428e0a2a34', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'bc7bf9fb-977a-4015-8739-6fa66c1ec210', 'attached_at': '', 'detached_at': '', 'volume_id': '7378e388-0e78-4212-b0ce-a1e082ba31ab', 'serial': '7378e388-0e78-4212-b0ce-a1e082ba31ab'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:14:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:24 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3b79f3b-6a47-4a3c-9c8d-10136178b174]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:14:24 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f13f0a1-90f0-4380-9464-f24baf43404a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab] added
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[41bbc9be-3ee6-4ae1-ab2b-43869a737563]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4127381d-cf84-46a9-8d3f-945499e959a2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tUxPOhHN {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tUxPOhHN" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bea4648a-7f8d-4add-805a-736ed62586d9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tUxPOhHN'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fd1d323-46dc-44d7-be41-39ef9ba8d549]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f1a9a08-569f-4f69-a13e-aa0d3b691df7]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --login" returned: 0 in 0.066s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[908bac1c-2f3a-4c0e-88f6-fdfff80d3ce3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]
Apr 14 03:14:24 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[190cffc0-644b-4f3d-8f82-d2082a86b7bc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5edfd5a-7699-47ab-b8c1-fff5f0af2b19]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:24 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:14:24 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.289s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('35', '-', '-', 0) found on session 37 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[014d046f-c542-4fe0-a827-fc1c1617f833]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 37 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:14:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.326s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1328ms) {'type': 'block', 'scsi_wwn': '23031383739646233', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23031383739646233', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:14:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:14:25 user nova-compute[71323]:   <uuid>bc7bf9fb-977a-4015-8739-6fa66c1ec210</uuid>
Apr 14 03:14:25 user nova-compute[71323]:   <name>instance-00000023</name>
Apr 14 03:14:25 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:14:25 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:14:25 user nova-compute[71323]:   <metadata>
Apr 14 03:14:25 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:14:25 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:14:25 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-463419112</nova:name>
Apr 14 03:14:25 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:14:24</nova:creationTime>
Apr 14 03:14:25 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:14:25 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:14:25 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:14:25 user nova-compute[71323]:       <nova:owner>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:14:25 user nova-compute[71323]:       </nova:owner>
Apr 14 03:14:25 user nova-compute[71323]:       <nova:ports>
Apr 14 03:14:25 user nova-compute[71323]:         <nova:port uuid="f1dbc6da-e8f9-4962-ac4c-db69105689f9">
Apr 14 03:14:25 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:14:25 user nova-compute[71323]:         </nova:port>
Apr 14 03:14:25 user nova-compute[71323]:       </nova:ports>
Apr 14 03:14:25 user nova-compute[71323]:     </nova:instance>
Apr 14 03:14:25 user nova-compute[71323]:   </metadata>
Apr 14 03:14:25 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:14:25 user nova-compute[71323]:     <system>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="serial">bc7bf9fb-977a-4015-8739-6fa66c1ec210</entry>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="uuid">bc7bf9fb-977a-4015-8739-6fa66c1ec210</entry>
Apr 14 03:14:25 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:14:25 user nova-compute[71323]:     </system>
Apr 14 03:14:25 user nova-compute[71323]:   </sysinfo>
Apr 14 03:14:25 user nova-compute[71323]:   <os>
Apr 14 03:14:25 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:14:25 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:14:25 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:14:25 user nova-compute[71323]:   </os>
Apr 14 03:14:25 user nova-compute[71323]:   <features>
Apr 14 03:14:25 user nova-compute[71323]:     <acpi/>
Apr 14 03:14:25 user nova-compute[71323]:     <apic/>
Apr 14 03:14:25 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:14:25 user nova-compute[71323]:   </features>
Apr 14 03:14:25 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:14:25 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:14:25 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:14:25 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:14:25 user nova-compute[71323]:   </clock>
Apr 14 03:14:25 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:14:25 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:14:25 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:14:25 user nova-compute[71323]:   </cpu>
Apr 14 03:14:25 user nova-compute[71323]:   <devices>
Apr 14 03:14:25 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:14:25 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:14:25 user nova-compute[71323]:       <source dev="/dev/sdd"/>
Apr 14 03:14:25 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:14:25 user nova-compute[71323]:       <serial>7378e388-0e78-4212-b0ce-a1e082ba31ab</serial>
Apr 14 03:14:25 user nova-compute[71323]:     </disk>
Apr 14 03:14:25 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:14:25 user nova-compute[71323]:       <mac address="fa:16:3e:b1:de:25"/>
Apr 14 03:14:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:14:25 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:14:25 user nova-compute[71323]:       <target dev="tapf1dbc6da-e8"/>
Apr 14 03:14:25 user nova-compute[71323]:     </interface>
Apr 14 03:14:25 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:14:25 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210/console.log" append="off"/>
Apr 14 03:14:25 user nova-compute[71323]:     </serial>
Apr 14 03:14:25 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:14:25 user nova-compute[71323]:     <video>
Apr 14 03:14:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:14:25 user nova-compute[71323]:     </video>
Apr 14 03:14:25 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:14:25 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:14:25 user nova-compute[71323]:     </rng>
Apr 14 03:14:25 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:14:25 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:14:25 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:14:25 user nova-compute[71323]:     </memballoon>
Apr 14 03:14:25 user nova-compute[71323]:   </devices>
Apr 14 03:14:25 user nova-compute[71323]: </domain>
Apr 14 03:14:25 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:14:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG os_vif [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf1dbc6da-e8, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf1dbc6da-e8, col_values=(('external_ids', {'iface-id': 'f1dbc6da-e8f9-4962-ac4c-db69105689f9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b1:de:25', 'vm-uuid': 'bc7bf9fb-977a-4015-8739-6fa66c1ec210'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:25 user nova-compute[71323]: INFO os_vif [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8')
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:14:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:b1:de:25, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:27 user nova-compute[71323]: WARNING nova.compute.manager [req-25c524ef-c7e7-4630-b25b-96fc6a57c7d3 req-3c7f162f-b8eb-4f7d-aa8a-c30bc8fbea85 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state building and task_state spawning.
Apr 14 03:14:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442069.3564775, bc7bf9fb-977a-4015-8739-6fa66c1ec210 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] VM Resumed (Lifecycle Event)
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance spawned successfully.
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442069.3589802, bc7bf9fb-977a-4015-8739-6fa66c1ec210 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] VM Started (Lifecycle Event)
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Took 4.97 seconds to spawn the instance on the hypervisor.
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:29 user nova-compute[71323]: WARNING nova.compute.manager [req-9c7cf017-286d-4da6-ad37-d9f344b6b18f req-ef7a0b93-0bfa-41ab-8fe6-775d19d00461 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state building and task_state spawning.
Apr 14 03:14:29 user nova-compute[71323]: INFO nova.compute.manager [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Took 10.45 seconds to build instance.
Apr 14 03:14:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5143b4d4-ae15-4d5c-a126-ae638aefefcb tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.552s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing instance network info cache due to event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] Acquiring lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] Acquired lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updated VIF entry in instance network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating instance_info_cache with network_info: [{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.51", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be61977d-6d8f-4ad9-b1e1-f727b96ef9e7 req-e5f7b6d4-e02b-4378-8c03-a4daac7c8d40 service nova] Releasing lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing instance network info cache due to event network-changed-2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] Acquiring lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] Acquired lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Refreshing network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:45 user nova-compute[71323]: INFO nova.compute.manager [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Terminating instance
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Instance destroyed successfully.
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid 5d0f51bd-b28d-4db1-be36-cec6cc92231e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1851758576',display_name='tempest-TestVolumeBootPattern-server-1851758576',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1851758576',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:13:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-engfvqb5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:13:03Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=5d0f51bd-b28d-4db1-be36-cec6cc92231e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.51", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.51", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG os_vif [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2885c06a-4d, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:45 user nova-compute[71323]: INFO os_vif [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:22:99:71,bridge_name='br-int',has_traffic_filtering=True,id=2885c06a-4dea-452f-a76a-e6a12bb525cc,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2885c06a-4d')
Apr 14 03:14:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tyovbWPn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:14:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:14:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:14:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b90123e7-faeb-438b-aa13-2ae2d9b5344a]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:14:46 user nova-compute[71323]: No targets found.
Apr 14 03:14:46 user nova-compute[71323]: iSNS:
Apr 14 03:14:46 user nova-compute[71323]: No targets found.
Apr 14 03:14:46 user nova-compute[71323]: STATIC:
Apr 14 03:14:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:14:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53
Apr 14 03:14:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:14:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:14:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:46 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:14:46 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:14:46 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:14:46 user nova-compute[71323]: FIRMWARE:
Apr 14 03:14:46 user nova-compute[71323]: No targets found.
Apr 14 03:14:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:14:46 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:14:46 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ff41e37-83e1-4d79-b285-35b69890dba1]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a69d87a-3c9e-4430-8f25-288314a581ac]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:14:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:14:46 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:14:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00613490>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[40a7a7c0-21e0-4175-88f7-0333be4f5e53]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2402c2c8-b2b1-4ee2-8988-bd6357adbc7a]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8892d0e8-d205-4b6a-baa2-20045804f962]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.068s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[57927713-ec8d-4e31-b5e5-17dfad2e664f]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c05c6cc7-d505-4f17-bde1-101568d52aa0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5022346-a9ec-4700-a768-976b75fa9296]: (4, ('Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260]\nLogout of [sid: 34, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260]
Apr 14 03:14:46 user nova-compute[71323]: Logout of [sid: 34, target: iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53, portal: 172.16.0.220,3260] successful.
Apr 14 03:14:46 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0ae128f0-64ab-4413-bae2-15cd90212c53 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4007bce9-7354-4450-bc9d-3e67ade706a7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (363ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:14:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Deleting instance files /opt/stack/data/nova/instances/5d0f51bd-b28d-4db1-be36-cec6cc92231e_del
Apr 14 03:14:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Deletion of /opt/stack/data/nova/instances/5d0f51bd-b28d-4db1-be36-cec6cc92231e_del complete
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updated VIF entry in instance network info cache for port 2885c06a-4dea-452f-a76a-e6a12bb525cc. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating instance_info_cache with network_info: [{"id": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "address": "fa:16:3e:22:99:71", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2885c06a-4d", "ovs_interfaceid": "2885c06a-4dea-452f-a76a-e6a12bb525cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af424a8-4bc9-421e-8f91-a3c67156f839 req-e664e2a9-1ec1-4478-94a3-581820595f85 service nova] Releasing lock "refresh_cache-5d0f51bd-b28d-4db1-be36-cec6cc92231e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:14:46 user nova-compute[71323]: INFO nova.compute.manager [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Took 1.25 seconds to destroy the instance on the hypervisor.
Apr 14 03:14:46 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:46 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Took 0.51 seconds to deallocate network for instance.
Apr 14 03:14:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-29f26d0e-2884-48ec-b03e-a2969848e3b9 req-ab6afeef-d093-4b3e-bc2b-0b8cd9bb6ce3 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-deleted-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-unplugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] No waiting events found dispatching network-vif-unplugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-unplugged-2885c06a-4dea-452f-a76a-e6a12bb525cc for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Acquiring lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] No waiting events found dispatching network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:47 user nova-compute[71323]: WARNING nova.compute.manager [req-0efea8b1-cbcb-44f3-8b08-e563e79d2ad5 req-cc9e9edb-00aa-4640-a1a5-4eb734c4d507 service nova] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Received unexpected event network-vif-plugged-2885c06a-4dea-452f-a76a-e6a12bb525cc for instance with vm_state active and task_state deleting.
Apr 14 03:14:47 user nova-compute[71323]: INFO nova.compute.manager [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Took 0.76 seconds to detach 1 volumes for instance.
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:48 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:48 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.338s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:48 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance 5d0f51bd-b28d-4db1-be36-cec6cc92231e
Apr 14 03:14:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5b56739b-53e5-4b67-ba3d-2d87b65d5e06 tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "5d0f51bd-b28d-4db1-be36-cec6cc92231e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.036s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:53 user nova-compute[71323]: INFO nova.compute.manager [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Terminating instance
Apr 14 03:14:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-da9ca23a-004a-478d-a14d-8b9451aff0cd req-32a148d3-1a01-49eb-8493-a3fb78270737 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Instance destroyed successfully.
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lazy-loading 'resources' on Instance uuid 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T02:57:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-846820438',display_name='tempest-ServerRescueNegativeTestJSON-server-846820438',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-846820438',id=3,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:02:48Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='558bfd7e1d9049d396381de148b0c4a5',ramdisk_id='',reservation_id='r-5u5lcg43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-773851452',owner_user_name='tempest-ServerRescueNegativeTestJSON-773851452-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:04:39Z,user_data=None,user_id='d5e8150f1502427fb3148f20354d212e',uuid=5e57cef1-b6cd-4291-a4c5-60e9dd22aed6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converting VIF {"id": "1319e09d-3a9f-404b-862d-37b385badb8d", "address": "fa:16:3e:f3:f8:fa", "network": {"id": "d3496999-3ffc-4589-94fc-0795323a1d70", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-814238130-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "558bfd7e1d9049d396381de148b0c4a5", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1319e09d-3a", "ovs_interfaceid": "1319e09d-3a9f-404b-862d-37b385badb8d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG os_vif [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1319e09d-3a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:54 user nova-compute[71323]: INFO os_vif [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f3:f8:fa,bridge_name='br-int',has_traffic_filtering=True,id=1319e09d-3a9f-404b-862d-37b385badb8d,network=Network(d3496999-3ffc-4589-94fc-0795323a1d70),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1319e09d-3a')
Apr 14 03:14:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Deleting instance files /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6_del
Apr 14 03:14:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Deletion of /opt/stack/data/nova/instances/5e57cef1-b6cd-4291-a4c5-60e9dd22aed6_del complete
Apr 14 03:14:54 user nova-compute[71323]: INFO nova.compute.manager [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 03:14:54 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:14:54 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:14:55 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Took 0.66 seconds to deallocate network for instance.
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-e428638c-a328-43e8-a56e-e44463f3b8d5 req-69b31838-ac17-49c3-9e4d-0bcfaedd756d service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-deleted-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.334s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Deleted allocations for instance 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3c054b53-f863-4c88-a4a1-d1ae63c5330b tempest-ServerRescueNegativeTestJSON-773851452 tempest-ServerRescueNegativeTestJSON-773851452-project-member] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.065s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:55 user nova-compute[71323]: WARNING nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state deleted and task_state None.
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:55 user nova-compute[71323]: WARNING nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state deleted and task_state None.
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:55 user nova-compute[71323]: WARNING nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state deleted and task_state None.
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:55 user nova-compute[71323]: WARNING nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-unplugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state deleted and task_state None.
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Acquiring lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] Lock "5e57cef1-b6cd-4291-a4c5-60e9dd22aed6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] No waiting events found dispatching network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:55 user nova-compute[71323]: WARNING nova.compute.manager [req-42ed1a6b-9e94-4fef-80ea-cbbfd385b49e req-3ad96b54-2d6f-4319-b7d2-c6c16c702e95 service nova] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Received unexpected event network-vif-plugged-1319e09d-3a9f-404b-862d-37b385badb8d for instance with vm_state deleted and task_state None.
Apr 14 03:14:57 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:59 user nova-compute[71323]: INFO nova.compute.manager [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Terminating instance
Apr 14 03:14:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-unplugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] No waiting events found dispatching network-vif-unplugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-7807e2c4-6569-4d59-b842-773f5f26f248 req-bd4e7366-40ec-4171-b42f-3ba3e29e6e32 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-unplugged-2749fb25-d2ff-43bb-b58c-f1d7694570de for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:14:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Instance destroyed successfully.
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lazy-loading 'resources' on Instance uuid d309dc50-d3b0-47a4-b263-a67ae96e2f5b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2095269608',display_name='tempest-VolumesActionsTest-instance-2095269608',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2095269608',id=32,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:13:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1318728a20064d49b9c1b2f89175a364',ramdisk_id='',reservation_id='r-cdvmvmi5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-121883605',owner_user_name='tempest-VolumesActionsTest-121883605-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:13:16Z,user_data=None,user_id='b2facbf385e8402c86706cc28d4707af',uuid=d309dc50-d3b0-47a4-b263-a67ae96e2f5b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converting VIF {"id": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "address": "fa:16:3e:6c:a1:1d", "network": {"id": "5c0957e8-ebf2-4e1b-82f8-370034962c6d", "bridge": "br-int", "label": "tempest-VolumesActionsTest-2078225960-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "1318728a20064d49b9c1b2f89175a364", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2749fb25-d2", "ovs_interfaceid": "2749fb25-d2ff-43bb-b58c-f1d7694570de", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG os_vif [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2749fb25-d2, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:15:00 user nova-compute[71323]: INFO os_vif [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:a1:1d,bridge_name='br-int',has_traffic_filtering=True,id=2749fb25-d2ff-43bb-b58c-f1d7694570de,network=Network(5c0957e8-ebf2-4e1b-82f8-370034962c6d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2749fb25-d2')
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Deleting instance files /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b_del
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Deletion of /opt/stack/data/nova/instances/d309dc50-d3b0-47a4-b263-a67ae96e2f5b_del complete
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.compute.manager [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 03:15:00 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Took 0.53 seconds to deallocate network for instance.
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-59ebf3bc-0b68-4280-a732-d6adef40658d req-d50b5033-126d-42af-a784-a1d9d9290c45 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-deleted-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.compute.manager [req-59ebf3bc-0b68-4280-a732-d6adef40658d req-d50b5033-126d-42af-a784-a1d9d9290c45 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Neutron deleted interface 2749fb25-d2ff-43bb-b58c-f1d7694570de; detaching it from the instance and deleting it from the info cache
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-59ebf3bc-0b68-4280-a732-d6adef40658d req-d50b5033-126d-42af-a784-a1d9d9290c45 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-59ebf3bc-0b68-4280-a732-d6adef40658d req-d50b5033-126d-42af-a784-a1d9d9290c45 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Detach interface failed, port_id=2749fb25-d2ff-43bb-b58c-f1d7694570de, reason: Instance d309dc50-d3b0-47a4-b263-a67ae96e2f5b could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442085.944611, 5d0f51bd-b28d-4db1-be36-cec6cc92231e => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] VM Stopped (Lifecycle Event)
Apr 14 03:15:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0fa12d80-7bb7-466e-8823-541d4cc3fb7c None None] [instance: 5d0f51bd-b28d-4db1-be36-cec6cc92231e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.301s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:01 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Deleted allocations for instance d309dc50-d3b0-47a4-b263-a67ae96e2f5b
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6d5c9019-1ec9-43df-9e8b-5ad7be98530a tempest-VolumesActionsTest-121883605 tempest-VolumesActionsTest-121883605-project-member] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.896s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] Acquiring lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] Lock "d309dc50-d3b0-47a4-b263-a67ae96e2f5b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] No waiting events found dispatching network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:01 user nova-compute[71323]: WARNING nova.compute.manager [req-0c40fc34-d7c1-4bce-a25e-7b92725ae162 req-34bbeb8f-1d52-4810-9bdb-f41675b47114 service nova] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Received unexpected event network-vif-plugged-2749fb25-d2ff-43bb-b58c-f1d7694570de for instance with vm_state deleted and task_state None.
Apr 14 03:15:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:15:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:15:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.207s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk.rescue --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442094.420349, 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:09 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] VM Stopped (Lifecycle Event)
Apr 14 03:15:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e6246551-afe7-4c9d-8faf-b40127a8acd4 None None] [instance: 5e57cef1-b6cd-4291-a4c5-60e9dd22aed6] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:15:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7816MB free_disk=26.332847595214844GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 7f542934-641f-405f-811c-3601a3be389b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:15:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.585s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:15:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:15:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:15:15 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442100.0604606, d309dc50-d3b0-47a4-b263-a67ae96e2f5b => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:15 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] VM Stopped (Lifecycle Event)
Apr 14 03:15:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-45860f66-e561-4d6e-8ba5-1894cbae3502 None None] [instance: d309dc50-d3b0-47a4-b263-a67ae96e2f5b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:15:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:23 user nova-compute[71323]: INFO nova.compute.manager [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] instance snapshotting
Apr 14 03:15:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Beginning live snapshot process
Apr 14 03:15:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json -f qcow2" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json -f qcow2 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json -f qcow2" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6.delta 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6.delta 1073741824" returned: 0 in 0.049s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Quiescing instance not available: QEMU guest agent is not enabled.
Apr 14 03:15:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] COPY block job progress, current cursor: 0 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:15:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] COPY block job progress, current cursor: 43778048 final cursor: 43778048 {{(pid=71323) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Apr 14 03:15:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Skipping quiescing instance: QEMU guest agent is not enabled.
Apr 14 03:15:25 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:15:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6.delta /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6.delta /opt/stack/data/nova/instances/snapshots/tmp_ctobo8b/99709b34c82845ad8b3de5d08a0ca5f6" returned: 0 in 0.260s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Snapshot extracted, beginning image upload
Apr 14 03:15:28 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Snapshot image upload complete
Apr 14 03:15:28 user nova-compute[71323]: INFO nova.compute.manager [None req-35f41c24-c38d-4f02-a5a3-385eca5733ab tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Took 4.89 seconds to snapshot the instance on the hypervisor.
Apr 14 03:15:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing instance network info cache due to event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] Acquiring lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] Acquired lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:15:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.067s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:36 user nova-compute[71323]: INFO nova.compute.manager [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attaching volume d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e to /dev/vdb
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.utils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f38d0099-85b8-4ea3-a826-44099496b242]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[30325a01-9143-4bc6-a5ef-15eec1ee846a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[67dfad28-fac3-4392-894c-4625809f462f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4678762-2ab6-452b-a67f-a34f74e40aed]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updated VIF entry in instance network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating instance_info_cache with network_info: [{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.179", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:15:36 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d88c4f0-d517-456a-a5f2-8f6a4c2b8220]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG os_brick.utils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== get_connector_properties: return (258ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating existing volume attachment record: f43baa05-9a69-4e42-9354-87cf3649e8f6 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:15:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-612f766d-7888-4823-8b8e-55ab36be665f req-8a9e6e55-0a8f-4474-8698-1e1f75e76755 service nova] Releasing lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QsIZOXnq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:15:41 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:15:41 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:15:41 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[643871f9-3ec2-428a-be14-208eb32f4868]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:15:41 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaa2fd96-2420-444c-a500-6f2a6e5f8451]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e] added
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[406d50ae-c682-47a0-835c-992acda5d5b5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9ea5fa8-7c49-4db5-a1b2-92c87880cdd0]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QsIZOXnq {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v QsIZOXnq" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb3d69eb-83c4-42ea-b05f-c0c194bcbf61]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'QsIZOXnq'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6eebeb68-52f9-4c68-a52b-5faf450eb848]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc1e9149-87ad-472c-b832-5ef6f1b8f5c0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --login" returned: 0 in 0.079s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d41bd8e-fb3d-413c-9063-669c32fc0e05]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260]
Apr 14 03:15:41 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260] successful.
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e71ce60-b2b4-4e1c-adbe-f196838b6475]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1e094cc-387b-47ea-8f13-1550a7b0fda5]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:41 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:15:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.315s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] HCTL ('33', '-', '-', 0) found on session 38 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ba952c0-632d-424a-a730-d6407baab733]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 38 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:15:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QsIZOXnq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.357s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== connect_volume: return (1360ms) {'type': 'block', 'scsi_wwn': '26466363764356137', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26466363764356137', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.objects.instance [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:15:42 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:15:42 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:15:42 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:15:42 user nova-compute[71323]:   <serial>d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e</serial>
Apr 14 03:15:42 user nova-compute[71323]: </disk>
Apr 14 03:15:42 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:15:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No VIF found with MAC fa:16:3e:dd:ec:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05135c56-cc6d-4210-8c64-d96fe03acb9c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.952s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing instance network info cache due to event network-changed-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] Acquiring lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] Acquired lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Refreshing network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:43 user nova-compute[71323]: INFO nova.compute.manager [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Terminating instance
Apr 14 03:15:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-unplugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] No waiting events found dispatching network-vif-unplugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f5d674d8-5a65-4eef-a633-f7e013351cb7 req-3480bf69-5a00-4d7a-babf-4c2cd08d027d service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-unplugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updated VIF entry in instance network info cache for port 1ccf0327-517f-45c5-a995-1b3a7fcfdaf9. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating instance_info_cache with network_info: [{"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45707cf8-4a27-4c5e-b136-92ff58ec86ce req-ae9262fa-7dee-4359-b3da-0a525cb97ac8 service nova] Releasing lock "refresh_cache-7f542934-641f-405f-811c-3601a3be389b" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] Instance destroyed successfully.
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lazy-loading 'resources' on Instance uuid 7f542934-641f-405f-811c-3601a3be389b {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1780392212',display_name='tempest-TestVolumeBootPattern-server-1780392212',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1780392212',id=28,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKBsGfTnZtORAtMl//kkVhwXLO6q/szIxAnXs+gkckXj87K6zFVRS8GVQwVtrdT+ALB5PZ6w19ZXaBNEZoStHoMq1JdVvJjvDhLnJ6fjJJYgW5mf6nPdJbsmeMcY5EC9qQ==',key_name='tempest-TestVolumeBootPattern-742486677',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:11:01Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='2d2e49e8e92e416faebf55e3d4e0e087',ramdisk_id='',reservation_id='r-33ksjg22',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-910285030',owner_user_name='tempest-TestVolumeBootPattern-910285030-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:11:01Z,user_data=None,user_id='60e11b7ab2914209b23bf7a53791da64',uuid=7f542934-641f-405f-811c-3601a3be389b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.57", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converting VIF {"id": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "address": "fa:16:3e:b5:5a:1f", "network": {"id": "ea0f3bca-5794-46e4-a270-53dbe99375bd", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-1760237320-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.57", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "2d2e49e8e92e416faebf55e3d4e0e087", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1ccf0327-51", "ovs_interfaceid": "1ccf0327-517f-45c5-a995-1b3a7fcfdaf9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_vif [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1ccf0327-51, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:15:44 user nova-compute[71323]: INFO os_vif [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b5:5a:1f,bridge_name='br-int',has_traffic_filtering=True,id=1ccf0327-517f-45c5-a995-1b3a7fcfdaf9,network=Network(ea0f3bca-5794-46e4-a270-53dbe99375bd),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1ccf0327-51')
Apr 14 03:15:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tekamKoO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:15:44 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e181c557-352b-487f-be2d-8ad2b077446b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:15:44 user nova-compute[71323]: No targets found.
Apr 14 03:15:44 user nova-compute[71323]: iSNS:
Apr 14 03:15:44 user nova-compute[71323]: No targets found.
Apr 14 03:15:44 user nova-compute[71323]: STATIC:
Apr 14 03:15:44 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e
Apr 14 03:15:44 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:15:44 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:15:44 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:15:44 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:15:44 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:15:44 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691
Apr 14 03:15:44 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:15:44 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:15:44 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:15:44 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:15:44 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:15:44 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:15:44 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:15:44 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:15:44 user nova-compute[71323]: FIRMWARE:
Apr 14 03:15:44 user nova-compute[71323]: No targets found.
Apr 14 03:15:44 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:15:44 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:15:44 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[40f10979-9893-4fcc-826e-29d81bd5b918]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4639e929-cfa6-42c9-a68b-ca882ec18c56]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:15:44 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:15:44 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:15:44 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00240ee0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0d5f92a-65e0-40a9-8c05-6d2d16eddde4]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[08913948-7425-422a-9422-17b967a89be3]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1373c9d0-5014-4858-be46-42deb3ab9432]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d254029a-ffff-49c0-b9f2-723cdd96e02f]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.037s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7aa5f439-e896-4646-b6e4-840d43903b00]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e49b0a53-b36a-4caf-9237-ed435f17ca71]: (4, ('Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]\nLogout of [sid: 33, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260]
Apr 14 03:15:45 user nova-compute[71323]: Logout of [sid: 33, target: iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691, portal: 172.16.0.220,3260] successful.
Apr 14 03:15:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c6fbfee2-70fa-477a-a761-8d092061e691 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdef52c6-3e8d-4cdc-a7de-53f69c9534a9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.384s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] <== disconnect_volume: return (386ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:15:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Deleting instance files /opt/stack/data/nova/instances/7f542934-641f-405f-811c-3601a3be389b_del
Apr 14 03:15:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Deletion of /opt/stack/data/nova/instances/7f542934-641f-405f-811c-3601a3be389b_del complete
Apr 14 03:15:45 user nova-compute[71323]: INFO nova.compute.manager [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Took 1.31 seconds to destroy the instance on the hypervisor.
Apr 14 03:15:45 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:15:45 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:46 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] Took 0.95 seconds to deallocate network for instance.
Apr 14 03:15:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] Acquiring lock "7f542934-641f-405f-811c-3601a3be389b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] Lock "7f542934-641f-405f-811c-3601a3be389b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] No waiting events found dispatching network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:46 user nova-compute[71323]: WARNING nova.compute.manager [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received unexpected event network-vif-plugged-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 for instance with vm_state active and task_state deleting.
Apr 14 03:15:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-73b1a046-9970-4801-9bc6-0138061542ad req-84b62bfc-94ab-4434-b4ab-46b80e4b27c4 service nova] [instance: 7f542934-641f-405f-811c-3601a3be389b] Received event network-vif-deleted-1ccf0327-517f-45c5-a995-1b3a7fcfdaf9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:49 user nova-compute[71323]: INFO nova.compute.manager [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] [instance: 7f542934-641f-405f-811c-3601a3be389b] Took 3.53 seconds to detach 1 volumes for instance.
Apr 14 03:15:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:49 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:15:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:15:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.346s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:50 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Deleted allocations for instance 7f542934-641f-405f-811c-3601a3be389b
Apr 14 03:15:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-841070a3-6c4e-44e1-9802-b5e49847370e tempest-TestVolumeBootPattern-910285030 tempest-TestVolumeBootPattern-910285030-project-member] Lock "7f542934-641f-405f-811c-3601a3be389b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 6.350s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:52 user nova-compute[71323]: INFO nova.compute.manager [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Unrescuing
Apr 14 03:15:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:15:52 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:52 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'flavor' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:53 user nova-compute[71323]: WARNING nova.compute.manager [req-36ed31be-a2c0-4e56-9a5a-03c1f0d6cbad req-0e1060dd-004e-40ec-abe2-f0f22e9cd650 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:15:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing instance network info cache due to event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:15:53 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance destroyed successfully.
Apr 14 03:15:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updated VIF entry in instance network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:15:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbc902c6-c96e-482b-836b-42016da98a50 req-1243a88f-e9eb-4e76-bfb7-72be0f80e77c service nova] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:55 user nova-compute[71323]: WARNING nova.compute.manager [req-81864fe0-3ea1-4621-9a52-a668ef75f8fe req-bf50385c-87d8-4497-8f7b-a8b6878ce56c service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:15:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing instance network info cache due to event network-changed-2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Refreshing network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:56 user nova-compute[71323]: DEBUG nova.network.neutron [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updated VIF entry in instance network info cache for port 2d081ef4-9756-4571-b28c-0d4582c58f0a. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:15:56 user nova-compute[71323]: DEBUG nova.network.neutron [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:15:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-443b5deb-6c2b-4973-b237-a1160763f7c7 req-c91c607b-15c5-4cf8-be7f-511b8c78b1a6 service nova] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:57 user nova-compute[71323]: WARNING nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:15:57 user nova-compute[71323]: WARNING nova.compute.manager [req-39db3312-2875-4b12-b78c-3caae6f2770a req-04cd8ffd-b206-4d61-8aad-ca6f23086365 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 8a67985b-1938-4b7d-b486-13f6811611f3 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442157.655854, 8a67985b-1938-4b7d-b486-13f6811611f3 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Resumed (Lifecycle Event)
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7d6c79c2-40de-4fac-8d46-ef73f54753c0 tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:15:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442157.6575565, 8a67985b-1938-4b7d-b486-13f6811611f3 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Started (Lifecycle Event)
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:15:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:15:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:15:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:59 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442144.6127262, 7f542934-641f-405f-811c-3601a3be389b => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:15:59 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 7f542934-641f-405f-811c-3601a3be389b] VM Stopped (Lifecycle Event)
Apr 14 03:15:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-56585032-0178-404a-ae28-4bf81aa2d5c2 None None] [instance: 7f542934-641f-405f-811c-3601a3be389b] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:15:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:15:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:15:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:00 user nova-compute[71323]: INFO nova.compute.manager [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Detaching volume d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e
Apr 14 03:16:00 user nova-compute[71323]: INFO nova.virt.block_device [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attempting to driver detach volume d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e from mountpoint /dev/vdb
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Attempting to detach device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:16:00 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:00 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:16:00 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:16:00 user nova-compute[71323]:   <serial>d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e</serial>
Apr 14 03:16:00 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:16:00 user nova-compute[71323]: </disk>
Apr 14 03:16:00 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:16:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config.
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:16:00 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:00 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:16:00 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:16:00 user nova-compute[71323]:   <serial>d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e</serial>
Apr 14 03:16:00 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:16:00 user nova-compute[71323]: </disk>
Apr 14 03:16:00 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442160.6041827, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:16:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config.
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'QsIZOXnq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b13a876-9f93-43af-aed5-f62e89b71592]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:16:00 user nova-compute[71323]: No targets found.
Apr 14 03:16:00 user nova-compute[71323]: iSNS:
Apr 14 03:16:00 user nova-compute[71323]: No targets found.
Apr 14 03:16:00 user nova-compute[71323]: STATIC:
Apr 14 03:16:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e
Apr 14 03:16:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:16:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169
Apr 14 03:16:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:16:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:16:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:00 user nova-compute[71323]: FIRMWARE:
Apr 14 03:16:00 user nova-compute[71323]: No targets found.
Apr 14 03:16:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:16:00 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:16:00 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[29bdb348-8610-484d-a9a9-9a7742c66519]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[10481dfb-f4b5-49af-9759-306ad2274ff3]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)
Apr 14 03:16:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e (non-flash)
Apr 14 03:16:00 user nova-compute[71323]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)
Apr 14 03:16:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae01b01f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3444d4c9-2258-42bb-9ab4-fc536891e5ca]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[82da80ae-7bc2-410b-8f18-5f45c449ffc1]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[097bdeed-5701-4fd8-9c04-6b2413e8120c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.050s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0176a101-34c9-4c83-a127-a3ed23a0d41c]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2d40ac8-baf6-4496-b30d-e51142b7e281]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1825c09c-19d9-4194-a8a6-8add1edf5843]: (4, ('Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260]\nLogout of [sid: 38, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260]
Apr 14 03:16:00 user nova-compute[71323]: Logout of [sid: 38, target: iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e, portal: 172.16.0.220,3260] successful.
Apr 14 03:16:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d51ab0f1-4bb3-4e37-bb8e-8c3199935e9e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7b8eec7-98e7-43ca-a4ba-ff286f21fb22]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.344s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== disconnect_volume: return (344ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:16:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-80f70d52-e94a-4139-a502-c86db45dbb4c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.850s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:16:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:05 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:16:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.154s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.161s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7962MB free_disk=25.91043472290039GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1408MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing instance network info cache due to event network-changed-6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Refreshing network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:16:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.474s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updated VIF entry in instance network info cache for port 6ea4c00e-5d4b-4375-9b8d-60177a6ec526. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4a34e80e-509d-4584-b905-68323d8d503b req-4caeb243-e93d-4a58-90f9-475704c03eb7 service nova] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Stashing vm_state: active {{(pid=71323) _prep_resize /opt/stack/nova/nova/compute/manager.py:5499}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-changed-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Refreshing instance network info cache due to event network-changed-f1dbc6da-e8f9-4962-ac4c-db69105689f9. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Refreshing network info cache for port f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [{"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-8a67985b-1938-4b7d-b486-13f6811611f3" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_requests' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:16:12 user nova-compute[71323]: INFO nova.compute.claims [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Claim successful on node user
Apr 14 03:16:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG nova.objects.instance [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'pci_devices' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:13 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating resource usage from migration 4501397c-105e-46dc-84ab-e95c07de7d95
Apr 14 03:16:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updated VIF entry in instance network info cache for port f1dbc6da-e8f9-4962-ac4c-db69105689f9. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG nova.network.neutron [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-68b11a3f-c9a1-4c29-945e-f6d99de3dd72 req-dce8dc2c-9ffc-4aa5-9085-c8dc30c59bbd service nova] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:16:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 1.215s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:13 user nova-compute[71323]: INFO nova.compute.manager [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Migrating
Apr 14 03:16:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:14 user nova-compute[71323]: DEBUG nova.network.neutron [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:16:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:14 user nova-compute[71323]: DEBUG nova.network.neutron [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Starting migrate_disk_and_power_off {{(pid=71323) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11460}}
Apr 14 03:16:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:16 user nova-compute[71323]: DEBUG nova.compute.manager [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:16 user nova-compute[71323]: WARNING nova.compute.manager [req-e7f7c324-5517-49b2-8695-2daafe6f66b1 req-66f427e1-c6b6-4ddb-8fb0-25f31dd288f6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:16:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance destroyed successfully.
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:14:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:16:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf1dbc6da-e8, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:16:17 user nova-compute[71323]: INFO os_vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8')
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:17 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2fd6400-03d1-4bb8-a717-196bfbe06675]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:16:17 user nova-compute[71323]: No targets found.
Apr 14 03:16:17 user nova-compute[71323]: iSNS:
Apr 14 03:16:17 user nova-compute[71323]: No targets found.
Apr 14 03:16:17 user nova-compute[71323]: STATIC:
Apr 14 03:16:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:16:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169
Apr 14 03:16:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:16:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:16:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:17 user nova-compute[71323]: FIRMWARE:
Apr 14 03:16:17 user nova-compute[71323]: No targets found.
Apr 14 03:16:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:16:17 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:16:17 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6b7e588-15f8-4635-808c-eebe6a0b5f9a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[65e9d7b0-6a70-4caf-9e58-3ace8d6125af]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)
Apr 14 03:16:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:17 user nova-compute[71323]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:117adce3-717a-4664-8cfa-be7d1f5d0169 (non-flash)
Apr 14 03:16:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae016f6d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.033s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[de119a92-8a64-4d5a-a6d2-2405449a7e81]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0417f850-d188-4634-b5ef-20b6f41f9f02]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a27d18c2-69c6-49ad-b2df-67465c1d26f6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3cbd5c3-76d7-4230-9952-0f7b68ed4ba8]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23031383739646233', '/dev/disk/by-id/scsi-1SCST_BIOe41848c-01879db36f93b921', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_e41848c') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[15727bbb-2b19-48a7-9eb3-0e454692218f]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[45f0c090-eddc-4512-ab51-10f7d31cb93a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --logout" returned: 0 in 0.068s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8876a74-2617-4f02-a5d4-5560a30ec187]: (4, ('Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]\nLogout of [sid: 37, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]
Apr 14 03:16:17 user nova-compute[71323]: Logout of [sid: 37, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.
Apr 14 03:16:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[23bf129b-682f-4e50-bd4f-09d2c1e4a817]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.367s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (368ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:16:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] skipping disk /dev/sdd (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:16:18 user nova-compute[71323]: DEBUG nova.compute.manager [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:18 user nova-compute[71323]: DEBUG nova.compute.manager [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:18 user nova-compute[71323]: WARNING nova.compute.manager [req-deeca419-4bf3-4e52-986c-38b6ec9790fd req-8a723668-0910-4ea6-b5ee-f9fa1c6f37ca service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state active and task_state resize_migrating.
Apr 14 03:16:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Port f1dbc6da-e8f9-4962-ac4c-db69105689f9 binding to destination host user is already ACTIVE {{(pid=71323) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3161}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG nova.network.neutron [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:16:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG nova.network.neutron [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c038a1f0-a3d2-438e-8578-374effd98b19]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6aeec34-50cd-407e-920d-46dc39270989]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7a0555c-1713-4f4d-a3f9-67ddd7c3e3c0]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[85182ef2-8f6c-4b3a-b554-bc16a9bd7189]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:20 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:16:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[21baf9c5-35d9-4127-abe4-0fd7365a412b]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:16:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== get_connector_properties: return (126ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Starting finish_migration {{(pid=71323) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11647}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:16:24 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Creating image(s)
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Ensure instance console log exists: /opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start _get_guest_xml network_info=[{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '43ac2eb3-9113-4367-a91a-8cd526f86d35', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'bc7bf9fb-977a-4015-8739-6fa66c1ec210', 'attached_at': '2023-04-14T03:16:25.000000', 'detached_at': '', 'volume_id': '7378e388-0e78-4212-b0ce-a1e082ba31ab', 'serial': '7378e388-0e78-4212-b0ce-a1e082ba31ab'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:16:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:21Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='84',id=12,is_public=True,memory_mb=192,name='m1.micro',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.objects.instance [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'vcpu_model' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:24 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:24 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:24 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:16:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1792cf7-c31e-4a59-8a2f-a86de14258f2]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:16:25 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad784f61-06a9-4c32-ba01-b85e6a00ed00]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab] added
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.082s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[775ee719-46a4-4848-8f6b-b5f6ced8050d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0d61de1-f255-42ef-a575-6f3aea45bfda]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tUxPOhHN {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.username -v tUxPOhHN" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9ac93c0-718c-4713-9f31-89bf47d983d5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'tUxPOhHN'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f2614f9-6e0b-499b-80e4-d379e2d45a60]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a8e7ac7-e885-470e-928a-f3168f3e1e55]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[585a5bd6-a484-4491-8850-91010f3d1d8c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]
Apr 14 03:16:25 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a71bd9b7-b8e6-4e4d-9fcb-3834b4a9fcdc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdd4d9d5-aa63-4449-8f26-d8d87fa22a5a]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.363s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] HCTL ('33', '-', '-', 0) found on session 40 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ea6ae22-6c31-41d1-903a-203212683a86]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 40 and hctl ['33', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:25 user nova-compute[71323]: INFO nova.compute.manager [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Detaching volume 7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:16:25 user nova-compute[71323]: INFO nova.virt.block_device [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Attempting to driver detach volume 7fff5d89-4516-4be1-9698-f088498889ff from mountpoint /dev/vdb
Apr 14 03:16:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Attempting to detach device vdb from instance 8a67985b-1938-4b7d-b486-13f6811611f3 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:16:25 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:25 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:16:25 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:16:25 user nova-compute[71323]:   <serial>7fff5d89-4516-4be1-9698-f088498889ff</serial>
Apr 14 03:16:25 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:16:25 user nova-compute[71323]: </disk>
Apr 14 03:16:25 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:16:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Successfully detached device vdb from instance 8a67985b-1938-4b7d-b486-13f6811611f3 from the persistent domain config.
Apr 14 03:16:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 8a67985b-1938-4b7d-b486-13f6811611f3 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:16:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:16:25 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:25 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:16:25 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:16:25 user nova-compute[71323]:   <serial>7fff5d89-4516-4be1-9698-f088498889ff</serial>
Apr 14 03:16:25 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:16:25 user nova-compute[71323]: </disk>
Apr 14 03:16:25 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Searching for a device in session 40 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.397s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== connect_volume: return (1399ms) {'type': 'block', 'scsi_wwn': '23031383739646233', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23031383739646233', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:14:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:16:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:16:26 user nova-compute[71323]:   <uuid>bc7bf9fb-977a-4015-8739-6fa66c1ec210</uuid>
Apr 14 03:16:26 user nova-compute[71323]:   <name>instance-00000023</name>
Apr 14 03:16:26 user nova-compute[71323]:   <memory>196608</memory>
Apr 14 03:16:26 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:16:26 user nova-compute[71323]:   <metadata>
Apr 14 03:16:26 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:16:26 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:16:26 user nova-compute[71323]:       <nova:name>tempest-ServerActionsTestJSON-server-463419112</nova:name>
Apr 14 03:16:26 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:16:24</nova:creationTime>
Apr 14 03:16:26 user nova-compute[71323]:       <nova:flavor name="m1.micro">
Apr 14 03:16:26 user nova-compute[71323]:         <nova:memory>192</nova:memory>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:16:26 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:16:26 user nova-compute[71323]:       <nova:owner>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:user uuid="7dea23743ba945c4a9d539b8e277c653">tempest-ServerActionsTestJSON-1848627478-project-member</nova:user>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:project uuid="4075885e26cb4adaad24d3ae6217fe35">tempest-ServerActionsTestJSON-1848627478</nova:project>
Apr 14 03:16:26 user nova-compute[71323]:       </nova:owner>
Apr 14 03:16:26 user nova-compute[71323]:       <nova:ports>
Apr 14 03:16:26 user nova-compute[71323]:         <nova:port uuid="f1dbc6da-e8f9-4962-ac4c-db69105689f9">
Apr 14 03:16:26 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Apr 14 03:16:26 user nova-compute[71323]:         </nova:port>
Apr 14 03:16:26 user nova-compute[71323]:       </nova:ports>
Apr 14 03:16:26 user nova-compute[71323]:     </nova:instance>
Apr 14 03:16:26 user nova-compute[71323]:   </metadata>
Apr 14 03:16:26 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:16:26 user nova-compute[71323]:     <system>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="serial">bc7bf9fb-977a-4015-8739-6fa66c1ec210</entry>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="uuid">bc7bf9fb-977a-4015-8739-6fa66c1ec210</entry>
Apr 14 03:16:26 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:16:26 user nova-compute[71323]:     </system>
Apr 14 03:16:26 user nova-compute[71323]:   </sysinfo>
Apr 14 03:16:26 user nova-compute[71323]:   <os>
Apr 14 03:16:26 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:16:26 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:16:26 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:16:26 user nova-compute[71323]:   </os>
Apr 14 03:16:26 user nova-compute[71323]:   <features>
Apr 14 03:16:26 user nova-compute[71323]:     <acpi/>
Apr 14 03:16:26 user nova-compute[71323]:     <apic/>
Apr 14 03:16:26 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:16:26 user nova-compute[71323]:   </features>
Apr 14 03:16:26 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:16:26 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:16:26 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:16:26 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:16:26 user nova-compute[71323]:   </clock>
Apr 14 03:16:26 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:16:26 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:16:26 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:16:26 user nova-compute[71323]:   </cpu>
Apr 14 03:16:26 user nova-compute[71323]:   <devices>
Apr 14 03:16:26 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:16:26 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:26 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:16:26 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:16:26 user nova-compute[71323]:       <serial>7378e388-0e78-4212-b0ce-a1e082ba31ab</serial>
Apr 14 03:16:26 user nova-compute[71323]:     </disk>
Apr 14 03:16:26 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:16:26 user nova-compute[71323]:       <mac address="fa:16:3e:b1:de:25"/>
Apr 14 03:16:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:16:26 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:16:26 user nova-compute[71323]:       <target dev="tapf1dbc6da-e8"/>
Apr 14 03:16:26 user nova-compute[71323]:     </interface>
Apr 14 03:16:26 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:16:26 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210/console.log" append="off"/>
Apr 14 03:16:26 user nova-compute[71323]:     </serial>
Apr 14 03:16:26 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:16:26 user nova-compute[71323]:     <video>
Apr 14 03:16:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:16:26 user nova-compute[71323]:     </video>
Apr 14 03:16:26 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:16:26 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:16:26 user nova-compute[71323]:     </rng>
Apr 14 03:16:26 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:16:26 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:16:26 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:16:26 user nova-compute[71323]:     </memballoon>
Apr 14 03:16:26 user nova-compute[71323]:   </devices>
Apr 14 03:16:26 user nova-compute[71323]: </domain>
Apr 14 03:16:26 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:14:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=MigrationContext,new_flavor=Flavor(12),node='user',numa_topology=None,old_flavor=Flavor(11),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:16:19Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestJSON-283671701-network", "vif_mac": "fa:16:3e:b1:de:25"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf1dbc6da-e8, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapf1dbc6da-e8, col_values=(('external_ids', {'iface-id': 'f1dbc6da-e8f9-4962-ac4c-db69105689f9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:b1:de:25', 'vm-uuid': 'bc7bf9fb-977a-4015-8739-6fa66c1ec210'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:26 user nova-compute[71323]: INFO os_vif [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8')
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442186.4838529, 8a67985b-1938-4b7d-b486-13f6811611f3 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:16:26 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Successfully detached device vdb from instance 8a67985b-1938-4b7d-b486-13f6811611f3 from the live domain config.
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'weNMmovz', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:26 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[164257d1-ce8d-4112-ba9f-74d97a40f96d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:16:26 user nova-compute[71323]: No targets found.
Apr 14 03:16:26 user nova-compute[71323]: iSNS:
Apr 14 03:16:26 user nova-compute[71323]: No targets found.
Apr 14 03:16:26 user nova-compute[71323]: STATIC:
Apr 14 03:16:26 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff
Apr 14 03:16:26 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:26 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:26 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:16:26 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:26 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:26 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:16:26 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:16:26 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:16:26 user nova-compute[71323]: FIRMWARE:
Apr 14 03:16:26 user nova-compute[71323]: No targets found.
Apr 14 03:16:26 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:16:26 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:16:26 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] No VIF found with MAC fa:16:3e:b1:de:25, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[69904598-e7f4-48cd-834a-eb579d6eedf8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f07ebce-0fd5-438e-bf93-a0f74e742fb9]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:26 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:26 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:26 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:26 user nova-compute[71323]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff (non-flash)
Apr 14 03:16:26 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:26 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae01b3520>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b861782-03bc-4021-8777-fcdaffa5e1f5]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b63bda4-c4af-465b-877d-76a22d08dc61]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f4c7e20-0242-466a-91c9-0bfdf3cb468d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.051s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e76237ec-8708-4116-85d9-a0f6e5b59428]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9f74419-3aa3-4ddd-b70d-737b88cd10e8]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --logout" returned: 0 in 0.080s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d92d368f-1416-48fe-8112-cfe4f6c46870]: (4, ('Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260]\nLogout of [sid: 36, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260]
Apr 14 03:16:26 user nova-compute[71323]: Logout of [sid: 36, target: iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff, portal: 172.16.0.220,3260] successful.
Apr 14 03:16:26 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7fff5d89-4516-4be1-9698-f088498889ff -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ddd51ff-b99f-4b02-bd79-5eff2cf6b19e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.361s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] <== disconnect_volume: return (362ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:16:27 user nova-compute[71323]: DEBUG nova.objects.instance [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'flavor' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-53e2dc12-049d-4b69-9624-055d130919eb tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.118s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:28 user nova-compute[71323]: WARNING nova.compute.manager [req-69f1326d-98d3-4845-abaa-3cdcb7a49423 req-cf3b2ee6-b486-4634-b6b2-f20cda39aa31 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state active and task_state resize_finish.
Apr 14 03:16:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for bc7bf9fb-977a-4015-8739-6fa66c1ec210 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442190.0166814, bc7bf9fb-977a-4015-8739-6fa66c1ec210 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] VM Resumed (Lifecycle Event)
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance running successfully.
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Failed to set time: agent not configured {{(pid=71323) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-776ea954-fcb1-4fd4-a43f-418774ad4286 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] finish_migration finished successfully. {{(pid=71323) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11742}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442190.019647, bc7bf9fb-977a-4015-8739-6fa66c1ec210 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] VM Started (Lifecycle Event)
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: resize_finish, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:30 user nova-compute[71323]: WARNING nova.compute.manager [req-1fe8be52-2108-4945-abc4-a16fe399ebe9 req-3037b1de-9263-4c24-ac11-d0385f93c3b6 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state active and task_state resize_finish.
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] During sync_power_state the instance has a pending task (resize_finish). Skip.
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.compute.claims [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Claim successful on node user
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.465s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:16:30 user nova-compute[71323]: INFO nova.virt.block_device [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Booting with volume-backed-image 98e6a247-250b-4bb5-a947-e7f93598d16e at /dev/vda
Apr 14 03:16:30 user nova-compute[71323]: DEBUG nova.policy [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd666936eb0ee43a2ad9e17a12e6e273d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd2c5b2aafaf4ca7bf62e988ac726658', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:16:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Successfully created port: eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Successfully updated port: eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-changed-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Refreshing instance network info cache due to event network-changed-eac1de6c-0025-4c84-a354-e1418e69740f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] Acquiring lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:16:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:16:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating instance_info_cache with network_info: [{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance network_info: |[{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:16:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] Acquired lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:16:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Refreshing network info cache for port eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:16:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updated VIF entry in instance network info cache for port eac1de6c-0025-4c84-a354-e1418e69740f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:16:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating instance_info_cache with network_info: [{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:16:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a85d9fff-5c23-4d39-b4d4-3ebf399a572b req-de500549-eb84-4247-af30-9e4065c9cad9 service nova] Releasing lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.utils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3a3771b-299b-4b2e-832a-ad58104f2c45]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0774258-0e5b-46d5-97b5-3949c4905406]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0c24ad3-ebdf-4393-8814-dc5b8e856ccf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[69273983-10ca-4290-b117-0f26add0fc03]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:36 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:16:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[358f0cab-e50c-4bac-952b-649075f6fe36]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG os_brick.utils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:36 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating existing volume attachment record: a5bbbb1b-897b-41af-817a-9723fbeb8547 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:16:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:16:41 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Creating image(s)
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Ensure instance console log exists: /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start _get_guest_xml network_info=[{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'a5bbbb1b-897b-41af-817a-9723fbeb8547', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ca85d0c8-d192-4bdf-b607-6fe637ad6f81', 'attached_at': '', 'detached_at': '', 'volume_id': '46511b99-537b-43ba-af8f-65b70b89170b', 'serial': '46511b99-537b-43ba-af8f-65b70b89170b'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:16:41 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:41 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:41 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:41 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:41 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[11e24aab-f89b-4d75-bc85-a00f5e9e4780]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:16:41 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7aee53f3-c0c1-42ed-8a9e-5934d7982897]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b] added
Apr 14 03:16:41 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c40b395-5370-4864-a163-574403a42a63]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[98223f86-3719-49fc-9488-c12eb844e6ab]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cwEYTBpt {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cwEYTBpt" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[91ab65ee-e8b4-4e8d-9732-817d5bcbcadf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cwEYTBpt'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[12b9a88b-d9cd-4f09-a08b-e66b25cf7819]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[093f2de1-5c36-4105-8784-4544edd4f728]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a437bd03-3772-4945-ae16-f06b037d88ba]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260]
Apr 14 03:16:42 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260] successful.
Apr 14 03:16:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a072b17f-c62b-4829-b128-61e81d8c5f52]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[58d4ba4d-de03-4099-b5cf-9a14d85fb4e0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:16:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:42 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:16:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.342s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] HCTL ('34', '-', '-', 0) found on session 41 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c267b43-39a0-4352-95ce-929f886352a7]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Searching for a device in session 41 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:16:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.388s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== connect_volume: return (1390ms) {'type': 'block', 'scsi_wwn': '23162323233663635', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23162323233663635', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:16:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-604221994',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-604221994',id=36,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-kjvjew37',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:16:31Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=ca85d0c8-d192-4bdf-b607-6fe637ad6f81,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.objects.instance [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:16:43 user nova-compute[71323]:   <uuid>ca85d0c8-d192-4bdf-b607-6fe637ad6f81</uuid>
Apr 14 03:16:43 user nova-compute[71323]:   <name>instance-00000024</name>
Apr 14 03:16:43 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:16:43 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:16:43 user nova-compute[71323]:   <metadata>
Apr 14 03:16:43 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:16:43 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-604221994</nova:name>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:16:41</nova:creationTime>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:16:43 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:16:43 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:owner>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:16:43 user nova-compute[71323]:       </nova:owner>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:16:43 user nova-compute[71323]:       <nova:ports>
Apr 14 03:16:43 user nova-compute[71323]:         <nova:port uuid="eac1de6c-0025-4c84-a354-e1418e69740f">
Apr 14 03:16:43 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:16:43 user nova-compute[71323]:         </nova:port>
Apr 14 03:16:43 user nova-compute[71323]:       </nova:ports>
Apr 14 03:16:43 user nova-compute[71323]:     </nova:instance>
Apr 14 03:16:43 user nova-compute[71323]:   </metadata>
Apr 14 03:16:43 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:16:43 user nova-compute[71323]:     <system>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="serial">ca85d0c8-d192-4bdf-b607-6fe637ad6f81</entry>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="uuid">ca85d0c8-d192-4bdf-b607-6fe637ad6f81</entry>
Apr 14 03:16:43 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:16:43 user nova-compute[71323]:     </system>
Apr 14 03:16:43 user nova-compute[71323]:   </sysinfo>
Apr 14 03:16:43 user nova-compute[71323]:   <os>
Apr 14 03:16:43 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:16:43 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:16:43 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:16:43 user nova-compute[71323]:   </os>
Apr 14 03:16:43 user nova-compute[71323]:   <features>
Apr 14 03:16:43 user nova-compute[71323]:     <acpi/>
Apr 14 03:16:43 user nova-compute[71323]:     <apic/>
Apr 14 03:16:43 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:16:43 user nova-compute[71323]:   </features>
Apr 14 03:16:43 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:16:43 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:16:43 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:16:43 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:16:43 user nova-compute[71323]:   </clock>
Apr 14 03:16:43 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:16:43 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:16:43 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:16:43 user nova-compute[71323]:   </cpu>
Apr 14 03:16:43 user nova-compute[71323]:   <devices>
Apr 14 03:16:43 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:16:43 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:16:43 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:16:43 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:16:43 user nova-compute[71323]:       <serial>46511b99-537b-43ba-af8f-65b70b89170b</serial>
Apr 14 03:16:43 user nova-compute[71323]:     </disk>
Apr 14 03:16:43 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:16:43 user nova-compute[71323]:       <mac address="fa:16:3e:02:a9:64"/>
Apr 14 03:16:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:16:43 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:16:43 user nova-compute[71323]:       <target dev="tapeac1de6c-00"/>
Apr 14 03:16:43 user nova-compute[71323]:     </interface>
Apr 14 03:16:43 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:16:43 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/console.log" append="off"/>
Apr 14 03:16:43 user nova-compute[71323]:     </serial>
Apr 14 03:16:43 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:16:43 user nova-compute[71323]:     <video>
Apr 14 03:16:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:16:43 user nova-compute[71323]:     </video>
Apr 14 03:16:43 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:16:43 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:16:43 user nova-compute[71323]:     </rng>
Apr 14 03:16:43 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:16:43 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:16:43 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:16:43 user nova-compute[71323]:     </memballoon>
Apr 14 03:16:43 user nova-compute[71323]:   </devices>
Apr 14 03:16:43 user nova-compute[71323]: </domain>
Apr 14 03:16:43 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:16:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-604221994',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-604221994',id=36,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-kjvjew37',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:16:31Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=ca85d0c8-d192-4bdf-b607-6fe637ad6f81,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG os_vif [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapeac1de6c-00, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapeac1de6c-00, col_values=(('external_ids', {'iface-id': 'eac1de6c-0025-4c84-a354-e1418e69740f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:02:a9:64', 'vm-uuid': 'ca85d0c8-d192-4bdf-b607-6fe637ad6f81'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:43 user nova-compute[71323]: INFO os_vif [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00')
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:16:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:02:a9:64, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:16:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:45 user nova-compute[71323]: WARNING nova.compute.manager [req-e8af2e7f-e262-43b8-9740-7470acc6d558 req-acdb79f3-e8a1-4fed-8af1-9a47784cf9c0 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state building and task_state spawning.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442207.087278, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Resumed (Lifecycle Event)
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance spawned successfully.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442207.0884743, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Started (Lifecycle Event)
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:16:47 user nova-compute[71323]: WARNING nova.compute.manager [req-b13e7657-4a9f-4958-a7fb-7e7d8159fcaa req-868bdd3c-88b8-4d72-8d5b-c41e975de0d1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state building and task_state spawning.
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Took 5.34 seconds to spawn the instance on the hypervisor.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:16:47 user nova-compute[71323]: INFO nova.compute.manager [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Took 17.18 seconds to build instance.
Apr 14 03:16:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-076fc63b-a749-44f2-93ec-0f5c9b727ece tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 17.322s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG nova.objects.instance [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.069s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:16:54 user nova-compute[71323]: INFO nova.compute.manager [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attaching volume 972ac1b0-8c90-4a59-a53a-92321048d947 to /dev/vdb
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.utils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[82afc347-14ab-4f36-92b6-9077e2daa04d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3735615b-7f5c-488c-82af-edc4875b1be4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8038c78d-da05-433d-b878-7f17c7b5ce7f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b28bbb3-e8e0-47e7-b396-44fc7b66e8d7]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:16:54 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:16:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e8200ad-22a5-41b4-b3f9-409e61b2ceb7]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG os_brick.utils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== get_connector_properties: return (150ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:16:54 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating existing volume attachment record: 06f40f10-3df7-4048-9faf-8ffd652cf3a9 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:16:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:58 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BqELIMHk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:16:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:16:59 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[999bfb11-0d45-4708-884e-a5f3715eaf3a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:16:59 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[002d7592-6713-4301-be5e-e04b85827908]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947] added
Apr 14 03:16:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aba82ec5-cff5-46b2-80ae-a9b0bf7f07f3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6905f5a-01ac-4c70-8eba-faf964c6a8dd]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BqELIMHk {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BqELIMHk" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[39400fe1-c01c-4d69-8593-1f1ef3221a17]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'BqELIMHk'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c56f8c35-5b8f-44a2-9422-c50f96bcefd9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c61bea51-e804-49cb-9436-c7e4148604c0]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:59 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:59 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:16:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:16:59 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:16:59 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:16:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:16:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --login" returned: 0 in 1.645s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d054b6f-e30f-4432-8eda-007932b4134e]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260]
Apr 14 03:17:01 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260] successful.
Apr 14 03:17:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a44d6d5-c2bf-4cf0-a2fa-410a839825eb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ce4c412-0bec-435f-88e4-533f4b8cec49]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)
Apr 14 03:17:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)
Apr 14 03:17:01 user nova-compute[71323]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)
Apr 14 03:17:01 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.913s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] HCTL ('36', '-', '-', 0) found on session 43 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[72d3a49d-c54f-45ff-9903-d5bd806d9341]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 43 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BqELIMHk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:17:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.952s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== connect_volume: return (2953ms) {'type': 'block', 'scsi_wwn': '26337646535303832', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26337646535303832', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:17:02 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:17:02 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:17:02 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:17:02 user nova-compute[71323]:   <serial>972ac1b0-8c90-4a59-a53a-92321048d947</serial>
Apr 14 03:17:02 user nova-compute[71323]: </disk>
Apr 14 03:17:02 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:17:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No VIF found with MAC fa:16:3e:dd:ec:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:17:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-14a668f1-51a5-4155-9442-ca06617a474c tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.652s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:04 user nova-compute[71323]: INFO nova.compute.manager [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Detaching volume 972ac1b0-8c90-4a59-a53a-92321048d947
Apr 14 03:17:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:17:04 user nova-compute[71323]: INFO nova.virt.block_device [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attempting to driver detach volume 972ac1b0-8c90-4a59-a53a-92321048d947 from mountpoint /dev/vdb
Apr 14 03:17:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Attempting to detach device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:17:04 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:17:04 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:17:04 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:17:04 user nova-compute[71323]:   <serial>972ac1b0-8c90-4a59-a53a-92321048d947</serial>
Apr 14 03:17:04 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:17:04 user nova-compute[71323]: </disk>
Apr 14 03:17:04 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:17:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config.
Apr 14 03:17:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:17:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:17:04 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:17:04 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:17:04 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:17:04 user nova-compute[71323]:   <serial>972ac1b0-8c90-4a59-a53a-92321048d947</serial>
Apr 14 03:17:04 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:17:04 user nova-compute[71323]: </disk>
Apr 14 03:17:04 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442225.2608986, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:17:05 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config.
Apr 14 03:17:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BqELIMHk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:17:05 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3e93635-212c-48a2-96b3-1a2328d7c062]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:17:05 user nova-compute[71323]: No targets found.
Apr 14 03:17:05 user nova-compute[71323]: iSNS:
Apr 14 03:17:05 user nova-compute[71323]: No targets found.
Apr 14 03:17:05 user nova-compute[71323]: STATIC:
Apr 14 03:17:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947
Apr 14 03:17:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:17:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:17:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a
Apr 14 03:17:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:17:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:17:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:17:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:17:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:17:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:17:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:17:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:17:05 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:17:05 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:17:05 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:17:05 user nova-compute[71323]: FIRMWARE:
Apr 14 03:17:05 user nova-compute[71323]: No targets found.
Apr 14 03:17:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:17:05 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:17:05 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b680eff7-4c0a-4d3a-8f4f-10ee021733b9]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[77e253b3-0253-4c3d-b3e9-17eb9908fce8]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)
Apr 14 03:17:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:abd46acc-408d-423d-9d2a-efe1b436150a (non-flash)
Apr 14 03:17:05 user nova-compute[71323]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 (non-flash)
Apr 14 03:17:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ac1e3d240>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8788f80-3e5e-4a8d-bc3c-5bed516cb71f]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9c6b601-426a-4384-9f09-93bc8c71992b]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[93d7a5b0-e439-4c56-8a2a-e35ed28007b7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.039s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[24c299b1-a379-4a54-83f2-77e9e49d465c]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[64c8a408-8447-43cc-8813-d72ffff59f86]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dc1c88d-fb7e-441b-a233-ecbded8b4c97]: (4, ('Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260]\nLogout of [sid: 43, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260]
Apr 14 03:17:05 user nova-compute[71323]: Logout of [sid: 43, target: iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947, portal: 172.16.0.220,3260] successful.
Apr 14 03:17:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:972ac1b0-8c90-4a59-a53a-92321048d947 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9415759-5063-4fc5-a47d-e00ace6584d3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.336s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== disconnect_volume: return (337ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:17:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:17:06 user nova-compute[71323]: DEBUG nova.objects.instance [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-442e2f08-8fb3-46db-9b74-7587defb548b tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.701s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:17:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3/disk --force-share --output=json" returned: 0 in 0.220s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:17:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:17:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7858MB free_disk=26.275909423828125GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Applying migration context for instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 as it has an incoming, in-progress migration 4501397c-105e-46dc-84ab-e95c07de7d95. Migration status is finished {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:950}}
Apr 14 03:17:11 user nova-compute[71323]: INFO nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating resource usage from migration 4501397c-105e-46dc-84ab-e95c07de7d95
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8a67985b-1938-4b7d-b486-13f6811611f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Migration 4501397c-105e-46dc-84ab-e95c07de7d95 is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 128}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1637}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:17:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.461s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:17:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:17:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:17:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:17:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:19 user nova-compute[71323]: INFO nova.compute.manager [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Terminating instance
Apr 14 03:17:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.compute.manager [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.compute.manager [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.compute.manager [req-0232379d-d057-42c0-ab3f-39f8878de445 req-ca3c2216-2562-442b-b54a-1d5c4eaa65d2 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-unplugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:17:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Instance destroyed successfully.
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.objects.instance [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lazy-loading 'resources' on Instance uuid 8a67985b-1938-4b7d-b486-13f6811611f3 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:10:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-196777223',display_name='tempest-ServerStableDeviceRescueTest-server-196777223',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-196777223',id=26,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7w1V2h51K5WeC0lNrY7+q4q2XqyPZ1TB9opotq+DNhc9r2qQzgCUPc3xsU2/ON8yD/igMXO0IAbOHg83C68rdtapd1/KHmCO4Nj4cZtEPuSMjanMb8ZhuWuQX+Z2NrHw==',key_name='tempest-keypair-1447416444',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:14:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='82eb4fbf55f04294a3d97800142bdca9',ramdisk_id='',reservation_id='r-pniwrgja',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1642713032',owner_user_name='tempest-ServerStableDeviceRescueTest-1642713032-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:15:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='3c683fe5636b49858857033797ff8e68',uuid=8a67985b-1938-4b7d-b486-13f6811611f3,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converting VIF {"id": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "address": "fa:16:3e:af:d1:f9", "network": {"id": "ccf9becc-b97a-4847-b44d-cf9b5b02306c", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-1259360867-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.186", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "82eb4fbf55f04294a3d97800142bdca9", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2d081ef4-97", "ovs_interfaceid": "2d081ef4-9756-4571-b28c-0d4582c58f0a", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG os_vif [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2d081ef4-97, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:17:20 user nova-compute[71323]: INFO os_vif [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:af:d1:f9,bridge_name='br-int',has_traffic_filtering=True,id=2d081ef4-9756-4571-b28c-0d4582c58f0a,network=Network(ccf9becc-b97a-4847-b44d-cf9b5b02306c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2d081ef4-97')
Apr 14 03:17:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Deleting instance files /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3_del
Apr 14 03:17:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Deletion of /opt/stack/data/nova/instances/8a67985b-1938-4b7d-b486-13f6811611f3_del complete
Apr 14 03:17:20 user nova-compute[71323]: INFO nova.compute.manager [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Took 1.02 seconds to destroy the instance on the hypervisor.
Apr 14 03:17:20 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:17:20 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:17:21 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Took 1.17 seconds to deallocate network for instance.
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-ae9697a7-fc41-4382-b543-9e6647680155 req-338b462a-f977-4c15-bed0-96168efac5e3 service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-deleted-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] Acquiring lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] Lock "8a67985b-1938-4b7d-b486-13f6811611f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] No waiting events found dispatching network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:17:21 user nova-compute[71323]: WARNING nova.compute.manager [req-06697d97-9ff9-4e61-9d95-5d515a6d304d req-70dba5ab-0b70-465b-9dc9-eb0a6a6b0c6f service nova] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Received unexpected event network-vif-plugged-2d081ef4-9756-4571-b28c-0d4582c58f0a for instance with vm_state deleted and task_state None.
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:17:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.302s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:22 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Deleted allocations for instance 8a67985b-1938-4b7d-b486-13f6811611f3
Apr 14 03:17:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54c09ed6-dc78-4d7e-bd0f-66718d5b7f4b tempest-ServerStableDeviceRescueTest-1642713032 tempest-ServerStableDeviceRescueTest-1642713032-project-member] Lock "8a67985b-1938-4b7d-b486-13f6811611f3" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.680s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:17:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:35 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442240.2417552, 8a67985b-1938-4b7d-b486-13f6811611f3 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:17:35 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] VM Stopped (Lifecycle Event)
Apr 14 03:17:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fc5a0fd5-143b-4a77-91a1-6117cc9b600b None None] [instance: 8a67985b-1938-4b7d-b486-13f6811611f3] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:17:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:47 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:17:47 user nova-compute[71323]: INFO nova.compute.claims [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Claim successful on node user
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.334s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:17:48 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.policy [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '12199a43298942898a2e694ee2c5230e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0bd451b3f6734fe3ac0a8e8a853bb9eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:17:48 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Creating image(s)
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.part --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.virt.images [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] 3af87ed2-d2c6-4608-b72f-8b0f91e4ca86 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:17:48 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.part /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Successfully created port: a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.part /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.converted" returned: 0 in 0.251s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f.converted --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.928s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f,backing_fmt=raw /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f,backing_fmt=raw /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk 1073741824" returned: 0 in 0.182s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "d163f2165f3760e6684a8f425d81d9c0fc425f9f" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.320s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:17:49 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Cannot resize image /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'migration_context' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Ensure instance console log exists: /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Successfully updated port: a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing instance network info cache due to event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance network_info: |[{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start _get_guest_xml network_info=[{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T03:17:45Z,direct_url=<?>,disk_format='qcow2',id=3af87ed2-d2c6-4608-b72f-8b0f91e4ca86,min_disk=0,min_ram=0,name='tempest-scenario-img--1917162322',owner='0bd451b3f6734fe3ac0a8e8a853bb9eb',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T03:17:47Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '3af87ed2-d2c6-4608-b72f-8b0f91e4ca86'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:17:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:17:50 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T03:17:45Z,direct_url=<?>,disk_format='qcow2',id=3af87ed2-d2c6-4608-b72f-8b0f91e4ca86,min_disk=0,min_ram=0,name='tempest-scenario-img--1917162322',owner='0bd451b3f6734fe3ac0a8e8a853bb9eb',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T03:17:47Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:17:48Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'pci_devices' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:17:50 user nova-compute[71323]:   <uuid>8cabfa40-3167-4f13-b6f3-4064d326f837</uuid>
Apr 14 03:17:50 user nova-compute[71323]:   <name>instance-00000025</name>
Apr 14 03:17:50 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:17:50 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:17:50 user nova-compute[71323]:   <metadata>
Apr 14 03:17:50 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:17:50 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:name>tempest-TestMinimumBasicScenario-server-252373748</nova:name>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:17:50</nova:creationTime>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:17:50 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:17:50 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:owner>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:user uuid="12199a43298942898a2e694ee2c5230e">tempest-TestMinimumBasicScenario-144187586-project-member</nova:user>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:project uuid="0bd451b3f6734fe3ac0a8e8a853bb9eb">tempest-TestMinimumBasicScenario-144187586</nova:project>
Apr 14 03:17:50 user nova-compute[71323]:       </nova:owner>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:root type="image" uuid="3af87ed2-d2c6-4608-b72f-8b0f91e4ca86"/>
Apr 14 03:17:50 user nova-compute[71323]:       <nova:ports>
Apr 14 03:17:50 user nova-compute[71323]:         <nova:port uuid="a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0">
Apr 14 03:17:50 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 03:17:50 user nova-compute[71323]:         </nova:port>
Apr 14 03:17:50 user nova-compute[71323]:       </nova:ports>
Apr 14 03:17:50 user nova-compute[71323]:     </nova:instance>
Apr 14 03:17:50 user nova-compute[71323]:   </metadata>
Apr 14 03:17:50 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:17:50 user nova-compute[71323]:     <system>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="serial">8cabfa40-3167-4f13-b6f3-4064d326f837</entry>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="uuid">8cabfa40-3167-4f13-b6f3-4064d326f837</entry>
Apr 14 03:17:50 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:17:50 user nova-compute[71323]:     </system>
Apr 14 03:17:50 user nova-compute[71323]:   </sysinfo>
Apr 14 03:17:50 user nova-compute[71323]:   <os>
Apr 14 03:17:50 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:17:50 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:17:50 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:17:50 user nova-compute[71323]:   </os>
Apr 14 03:17:50 user nova-compute[71323]:   <features>
Apr 14 03:17:50 user nova-compute[71323]:     <acpi/>
Apr 14 03:17:50 user nova-compute[71323]:     <apic/>
Apr 14 03:17:50 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:17:50 user nova-compute[71323]:   </features>
Apr 14 03:17:50 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:17:50 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:17:50 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:17:50 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:17:50 user nova-compute[71323]:   </clock>
Apr 14 03:17:50 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:17:50 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:17:50 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:17:50 user nova-compute[71323]:   </cpu>
Apr 14 03:17:50 user nova-compute[71323]:   <devices>
Apr 14 03:17:50 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:17:50 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:17:50 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk"/>
Apr 14 03:17:50 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:17:50 user nova-compute[71323]:     </disk>
Apr 14 03:17:50 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:17:50 user nova-compute[71323]:       <mac address="fa:16:3e:71:a9:a9"/>
Apr 14 03:17:50 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:17:50 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:17:50 user nova-compute[71323]:       <target dev="tapa478a9ea-0f"/>
Apr 14 03:17:50 user nova-compute[71323]:     </interface>
Apr 14 03:17:50 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:17:50 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/console.log" append="off"/>
Apr 14 03:17:50 user nova-compute[71323]:     </serial>
Apr 14 03:17:50 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:17:50 user nova-compute[71323]:     <video>
Apr 14 03:17:50 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:17:50 user nova-compute[71323]:     </video>
Apr 14 03:17:50 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:17:50 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:17:50 user nova-compute[71323]:     </rng>
Apr 14 03:17:50 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:17:50 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:17:50 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:17:50 user nova-compute[71323]:     </memballoon>
Apr 14 03:17:50 user nova-compute[71323]:   </devices>
Apr 14 03:17:50 user nova-compute[71323]: </domain>
Apr 14 03:17:50 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:17:48Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG os_vif [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa478a9ea-0f, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa478a9ea-0f, col_values=(('external_ids', {'iface-id': 'a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:71:a9:a9', 'vm-uuid': '8cabfa40-3167-4f13-b6f3-4064d326f837'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:50 user nova-compute[71323]: INFO os_vif [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f')
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:17:50 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No VIF found with MAC fa:16:3e:71:a9:a9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:17:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updated VIF entry in instance network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:17:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:17:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-da0fc2b0-f30e-44f0-861a-21463be5d18b req-4df353fc-08fd-4035-a7d4-6e9de4795dd3 service nova] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:17:52 user nova-compute[71323]: WARNING nova.compute.manager [req-c727691b-3448-4368-8fb9-f7939935d72a req-ce943d89-7b01-4695-a23a-d25557326178 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state building and task_state spawning.
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442274.2711053, 8cabfa40-3167-4f13-b6f3-4064d326f837 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] VM Resumed (Lifecycle Event)
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance spawned successfully.
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442274.272407, 8cabfa40-3167-4f13-b6f3-4064d326f837 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] VM Started (Lifecycle Event)
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Took 5.97 seconds to spawn the instance on the hypervisor.
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:17:54 user nova-compute[71323]: WARNING nova.compute.manager [req-874c23be-0821-4528-a9d8-2fa6fe4bc17c req-f7552ff4-b300-476d-abb0-9c185f16bd9e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state building and task_state spawning.
Apr 14 03:17:54 user nova-compute[71323]: INFO nova.compute.manager [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Took 6.60 seconds to build instance.
Apr 14 03:17:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7fda8bd9-36f3-495b-bb29-f9e78f939444 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.734s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.060s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:17:58 user nova-compute[71323]: INFO nova.compute.manager [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attaching volume e9d830c5-2948-4289-9100-8665b806d340 to /dev/vdb
Apr 14 03:17:58 user nova-compute[71323]: DEBUG os_brick.utils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:17:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ac92c3e-dbd2-4812-8e1b-92e568e79e6d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[516ba462-a222-4327-b84e-8198d03330da]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f92a764a-6f65-4ab7-b6df-c2ded7703c73]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0c2ec78-0544-4fa5-83bb-cd77c5c2ac13]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:17:59 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e942af6-c364-4925-adb2-966b45d6e376]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG os_brick.utils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== get_connector_properties: return (127ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating existing volume attachment record: a00befe5-f508-4de2-aa52-3dda9c41ed10 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:17:59 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" acquired by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Going to confirm migration 2 {{(pid=71323) do_confirm_resize /opt/stack/nova/nova/compute/manager.py:4663}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquired lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'info_cache' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [{"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Releasing lock "refresh_cache-bc7bf9fb-977a-4015-8739-6fa66c1ec210" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'migration_context' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: held 0.350s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:03 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Deleted allocation for migration 4501397c-105e-46dc-84ab-e95c07de7d95
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dcfaa72b-88d2-492c-a855-c0a8c362047c tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" "released" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: held 1.411s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dbfZOqJS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:18:03 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:18:03 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:18:03 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e22565bd-7720-4857-bc74-08d99c789d0d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:18:03 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c4f74e5-310f-4e03-8d81-d346e99c05f9]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340] added
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[70f89bfd-7d28-4c08-a2e0-3c2b17a94b7b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0984d8c-cf84-4e58-8e4a-515a99e0c3dd]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dbfZOqJS {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v dbfZOqJS" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6132ab8-c177-44e5-9cbc-f8c494717f9c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'dbfZOqJS'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b60754bb-acc8-4e62-bbb0-1cdd8bfe2d4a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a0a5a87-3c74-4d1b-8612-d376c19bc038]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --login" returned: 0 in 0.071s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfccb985-c1d1-48dc-9e25-35c10a3bc99f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260]
Apr 14 03:18:03 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260] successful.
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1df1d16-9ea6-4518-bcef-6402f1061007]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7844700f-f06b-43e1-9e0b-0d2fea683528]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:03 user nova-compute[71323]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)
Apr 14 03:18:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.328s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] HCTL ('35', '-', '-', 0) found on session 44 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:18:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18d13bc3-2b5d-4484-a815-c606c98b7c94]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 44 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:18:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 44 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dbfZOqJS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.360s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== connect_volume: return (1362ms) {'type': 'block', 'scsi_wwn': '26233643035346335', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26233643035346335', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:18:05 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:18:05 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:18:05 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:18:05 user nova-compute[71323]:   <serial>e9d830c5-2948-4289-9100-8665b806d340</serial>
Apr 14 03:18:05 user nova-compute[71323]: </disk>
Apr 14 03:18:05 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No VIF found with MAC fa:16:3e:dd:ec:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4cb0c5c5-0eb8-4be0-a9f7-7e32060b0c33 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.665s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:07 user nova-compute[71323]: INFO nova.compute.manager [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Detaching volume e9d830c5-2948-4289-9100-8665b806d340
Apr 14 03:18:07 user nova-compute[71323]: INFO nova.virt.block_device [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attempting to driver detach volume e9d830c5-2948-4289-9100-8665b806d340 from mountpoint /dev/vdb
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Attempting to detach device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:18:07 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:18:07 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:18:07 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:18:07 user nova-compute[71323]:   <serial>e9d830c5-2948-4289-9100-8665b806d340</serial>
Apr 14 03:18:07 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:18:07 user nova-compute[71323]: </disk>
Apr 14 03:18:07 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:18:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config.
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:18:07 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:18:07 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:18:07 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:18:07 user nova-compute[71323]:   <serial>e9d830c5-2948-4289-9100-8665b806d340</serial>
Apr 14 03:18:07 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:18:07 user nova-compute[71323]: </disk>
Apr 14 03:18:07 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442287.8596582, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:18:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config.
Apr 14 03:18:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'dbfZOqJS', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:18:07 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7ddd060-8baa-4170-81cf-424cc833a043]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:18:07 user nova-compute[71323]: No targets found.
Apr 14 03:18:07 user nova-compute[71323]: iSNS:
Apr 14 03:18:07 user nova-compute[71323]: No targets found.
Apr 14 03:18:07 user nova-compute[71323]: STATIC:
Apr 14 03:18:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340
Apr 14 03:18:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:18:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:18:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:18:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:18:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:18:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:18:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:18:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:18:07 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:18:07 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:18:07 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:18:07 user nova-compute[71323]: FIRMWARE:
Apr 14 03:18:07 user nova-compute[71323]: No targets found.
Apr 14 03:18:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:18:07 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:18:07 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4afdb3a4-4961-4210-b9e1-356aad7e17ec]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b270d32c-f3b3-490b-8ffb-850624031bf6]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)
Apr 14 03:18:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:07 user nova-compute[71323]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 (non-flash)
Apr 14 03:18:07 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b00240940>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f907f28f-ce7a-4901-80e1-023b59635e0e]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:07 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[780f755d-8f1f-4baa-ade9-1dbe183cb691]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[785460f3-58fe-48a4-a98b-8a48adf97843]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecae3b81-c10b-4f0a-818f-ed5eb4ed79f1]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[690d74e6-6c0d-47cc-b373-86ad6b06f91c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e4711aa-e5d2-4410-bb6f-7d1b9ef159dc]: (4, ('Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260]\nLogout of [sid: 44, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260]
Apr 14 03:18:08 user nova-compute[71323]: Logout of [sid: 44, target: iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340, portal: 172.16.0.220,3260] successful.
Apr 14 03:18:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:e9d830c5-2948-4289-9100-8665b806d340 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d1649f4-58f7-4a84-bb48-067ae19fe584]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.291s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== disconnect_volume: return (292ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4daf2b34-8ce5-4142-bc76-590d30180088 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.656s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.158s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8001MB free_disk=26.273719787597656GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8cabfa40-3167-4f13-b6f3-4064d326f837 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:18:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1600MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:18:11 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:18:11 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:18:11 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:18:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.385s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:18:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:21 user nova-compute[71323]: INFO nova.compute.manager [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Rescuing
Apr 14 03:18:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:18:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:18:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:18:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating instance_info_cache with network_info: [{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:22 user nova-compute[71323]: WARNING nova.compute.manager [req-e472632d-dff0-4290-9d6b-6ad22b9b4617 req-b04eadbe-6d7b-451c-a916-4675aab899f6 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state active and task_state rescuing.
Apr 14 03:18:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance destroyed successfully.
Apr 14 03:18:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Attempting a stable device rescue
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=71323) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4289}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance directory exists: not creating {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4694}}
Apr 14 03:18:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Creating image(s)
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.011s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'trusted_certs' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.part --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG nova.virt.images [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] 85fe392a-ee18-4801-b7f3-d1fefff51c4f was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:18:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.part /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:24 user nova-compute[71323]: WARNING nova.compute.manager [req-45c79053-f8bf-4362-9945-87ce86e51646 req-453c1a73-9d66-4b42-b579-a57916c45e79 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state active and task_state rescuing.
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.part /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.converted" returned: 0 in 0.871s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3.converted --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.913s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:24 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3 --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3,backing_fmt=raw /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3,backing_fmt=raw /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue" returned: 0 in 0.047s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "523c4ae7dc469830ffa38dc1f297039d8fb385f3" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.185s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'migration_context' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start _get_guest_xml network_info=[{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:02:a9:64"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '85fe392a-ee18-4801-b7f3-d1fefff51c4f', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'a5bbbb1b-897b-41af-817a-9723fbeb8547', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': 'ca85d0c8-d192-4bdf-b607-6fe637ad6f81', 'attached_at': '', 'detached_at': '', 'volume_id': '46511b99-537b-43ba-af8f-65b70b89170b', 'serial': '46511b99-537b-43ba-af8f-65b70b89170b'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': False, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'numa_topology' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'vcpu_model' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:18:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:18:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:18:25 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d90d2937-e9da-4e63-a375-963210c3faf7]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = cwEYTBpt\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Apr 14 03:18:25 user nova-compute[71323]: node.name = iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:18:25 user nova-compute[71323]: node.tpgt = -1
Apr 14 03:18:25 user nova-compute[71323]: node.startup = automatic
Apr 14 03:18:25 user nova-compute[71323]: node.leading_login = No
Apr 14 03:18:25 user nova-compute[71323]: iface.iscsi_ifacename = default
Apr 14 03:18:25 user nova-compute[71323]: iface.net_ifacename = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.ipaddress = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.prefix_len = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.hwaddress = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.transport_name = tcp
Apr 14 03:18:25 user nova-compute[71323]: iface.initiatorname = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.vlan_id = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.vlan_priority = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.vlan_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.iface_num = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.mtu = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.port = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.bootproto = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.subnet_mask = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.gateway = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_alt_client_id_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_alt_client_id = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_dns = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_learn_iqn = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_req_vendor_id_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_vendor_id_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_vendor_id = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.dhcp_slp_da = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.fragmentation = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.gratuitous_arp = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.incoming_forwarding = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.tos_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.tos = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.ttl = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.delayed_ack = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.tcp_nagle = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.tcp_wsf_state = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.tcp_wsf = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.tcp_timer_scale = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.tcp_timestamp = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.redirect = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.def_task_mgmt_timeout = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.header_digest = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.data_digest = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.immediate_data = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.initial_r2t = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.data_seq_inorder = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.data_pdu_inorder = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.erl = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.max_receive_data_len = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.first_burst_len = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.max_outstanding_r2t = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.max_burst_len = 0
Apr 14 03:18:25 user nova-compute[71323]: iface.chap_auth = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.bidi_chap = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.strict_login_compliance = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.discovery_auth = <empty>
Apr 14 03:18:25 user nova-compute[71323]: iface.discovery_logout = <empty>
Apr 14 03:18:25 user nova-compute[71323]: node.discovery_address = <empty>
Apr 14 03:18:25 user nova-compute[71323]: node.discovery_port = 0
Apr 14 03:18:25 user nova-compute[71323]: node.discovery_type = static
Apr 14 03:18:25 user nova-compute[71323]: node.session.initial_cmdsn = 0
Apr 14 03:18:25 user nova-compute[71323]: node.session.initial_login_retry_max = 8
Apr 14 03:18:25 user nova-compute[71323]: node.session.xmit_thread_priority = -20
Apr 14 03:18:25 user nova-compute[71323]: node.session.cmds_max = 128
Apr 14 03:18:25 user nova-compute[71323]: node.session.queue_depth = 32
Apr 14 03:18:25 user nova-compute[71323]: node.session.nr_sessions = 1
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.authmethod = CHAP
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.username = cwEYTBpt
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.password = ***
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.username_in = <empty>
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.password_in = <empty>
Apr 14 03:18:25 user nova-compute[71323]: node.session.auth.chap_algs = MD5
Apr 14 03:18:25 user nova-compute[71323]: node.session.timeo.replacement_timeout = 120
Apr 14 03:18:25 user nova-compute[71323]: node.session.err_timeo.abort_timeout = 15
Apr 14 03:18:25 user nova-compute[71323]: node.session.err_timeo.lu_reset_timeout = 30
Apr 14 03:18:25 user nova-compute[71323]: node.session.err_timeo.tgt_reset_timeout = 30
Apr 14 03:18:25 user nova-compute[71323]: node.session.err_timeo.host_reset_timeout = 60
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.FastAbort = Yes
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.InitialR2T = No
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.ImmediateData = Yes
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.FirstBurstLength = 262144
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.MaxBurstLength = 16776192
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.DefaultTime2Retain = 0
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.DefaultTime2Wait = 2
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.MaxConnections = 1
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.MaxOutstandingR2T = 1
Apr 14 03:18:25 user nova-compute[71323]: node.session.iscsi.ERL = 0
Apr 14 03:18:25 user nova-compute[71323]: node.session.scan = manual
Apr 14 03:18:25 user nova-compute[71323]: node.session.reopen_max = 0
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].address = 172.16.0.220
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].port = 3260
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].startup = manual
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].tcp.window_size = 524288
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].tcp.type_of_service = 0
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].timeo.logout_timeout = 15
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].timeo.login_timeout = 15
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].timeo.auth_timeout = 45
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].timeo.noop_out_interval = 5
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].timeo.noop_out_timeout = 5
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.HeaderDigest = None
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.DataDigest = None
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.IFMarker = No
Apr 14 03:18:25 user nova-compute[71323]: node.conn[0].iscsi.OFMarker = No
Apr 14 03:18:25 user nova-compute[71323]: # END RECORD
Apr 14 03:18:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d007eb7-94d5-4288-94bb-1097311ce4ca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6cca164-f93f-4298-870c-e778c1364688]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cwEYTBpt {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cwEYTBpt" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[231234a1-b1c9-4e3d-92ea-3b1c75390d97]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cwEYTBpt'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.030s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[58c2d768-274b-4dbf-8412-84f93b88a0cb]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[66ebfdd0-2353-4dd0-a7f5-02c9be128ef5]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:18:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:18:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:18:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.183s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] HCTL ('34', '0', '0', 0) found on session 41 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1771b3fe-b13a-4ac9-8c21-b04541c5121e]: (4, ('0 0 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Searching for a device in session 41 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.209s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== connect_volume: return (212ms) {'type': 'block', 'scsi_wwn': '23162323233663635', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23162323233663635', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:18:25 user nova-compute[71323]: INFO nova.compute.claims [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Claim successful on node user
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:16:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-604221994',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-604221994',id=36,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:16:47Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-kjvjew37',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:16:47Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=ca85d0c8-d192-4bdf-b607-6fe637ad6f81,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:02:a9:64"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "vif_mac": "fa:16:3e:02:a9:64"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.objects.instance [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'pci_devices' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:18:25 user nova-compute[71323]:   <uuid>ca85d0c8-d192-4bdf-b607-6fe637ad6f81</uuid>
Apr 14 03:18:25 user nova-compute[71323]:   <name>instance-00000024</name>
Apr 14 03:18:25 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:18:25 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:18:25 user nova-compute[71323]:   <metadata>
Apr 14 03:18:25 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:18:25 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-604221994</nova:name>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:18:25</nova:creationTime>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:18:25 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:18:25 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:owner>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:user uuid="d666936eb0ee43a2ad9e17a12e6e273d">tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member</nova:user>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:project uuid="bd2c5b2aafaf4ca7bf62e988ac726658">tempest-ServerBootFromVolumeStableRescueTest-1400694371</nova:project>
Apr 14 03:18:25 user nova-compute[71323]:       </nova:owner>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:18:25 user nova-compute[71323]:       <nova:ports>
Apr 14 03:18:25 user nova-compute[71323]:         <nova:port uuid="eac1de6c-0025-4c84-a354-e1418e69740f">
Apr 14 03:18:25 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:18:25 user nova-compute[71323]:         </nova:port>
Apr 14 03:18:25 user nova-compute[71323]:       </nova:ports>
Apr 14 03:18:25 user nova-compute[71323]:     </nova:instance>
Apr 14 03:18:25 user nova-compute[71323]:   </metadata>
Apr 14 03:18:25 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:18:25 user nova-compute[71323]:     <system>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="serial">ca85d0c8-d192-4bdf-b607-6fe637ad6f81</entry>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="uuid">ca85d0c8-d192-4bdf-b607-6fe637ad6f81</entry>
Apr 14 03:18:25 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:18:25 user nova-compute[71323]:     </system>
Apr 14 03:18:25 user nova-compute[71323]:   </sysinfo>
Apr 14 03:18:25 user nova-compute[71323]:   <os>
Apr 14 03:18:25 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:18:25 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:18:25 user nova-compute[71323]:   </os>
Apr 14 03:18:25 user nova-compute[71323]:   <features>
Apr 14 03:18:25 user nova-compute[71323]:     <acpi/>
Apr 14 03:18:25 user nova-compute[71323]:     <apic/>
Apr 14 03:18:25 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:18:25 user nova-compute[71323]:   </features>
Apr 14 03:18:25 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:18:25 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:18:25 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:18:25 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:18:25 user nova-compute[71323]:   </clock>
Apr 14 03:18:25 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:18:25 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:18:25 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:18:25 user nova-compute[71323]:   </cpu>
Apr 14 03:18:25 user nova-compute[71323]:   <devices>
Apr 14 03:18:25 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:18:25 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:18:25 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:18:25 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:18:25 user nova-compute[71323]:       <serial>46511b99-537b-43ba-af8f-65b70b89170b</serial>
Apr 14 03:18:25 user nova-compute[71323]:     </disk>
Apr 14 03:18:25 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:18:25 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:18:25 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue"/>
Apr 14 03:18:25 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:18:25 user nova-compute[71323]:       <boot order="1"/>
Apr 14 03:18:25 user nova-compute[71323]:     </disk>
Apr 14 03:18:25 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:18:25 user nova-compute[71323]:       <mac address="fa:16:3e:02:a9:64"/>
Apr 14 03:18:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:18:25 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:18:25 user nova-compute[71323]:       <target dev="tapeac1de6c-00"/>
Apr 14 03:18:25 user nova-compute[71323]:     </interface>
Apr 14 03:18:25 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:18:25 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/console.log" append="off"/>
Apr 14 03:18:25 user nova-compute[71323]:     </serial>
Apr 14 03:18:25 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:18:25 user nova-compute[71323]:     <video>
Apr 14 03:18:25 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:18:25 user nova-compute[71323]:     </video>
Apr 14 03:18:25 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:18:25 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:18:25 user nova-compute[71323]:     </rng>
Apr 14 03:18:25 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:18:25 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:18:25 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:18:25 user nova-compute[71323]:     </memballoon>
Apr 14 03:18:25 user nova-compute[71323]:   </devices>
Apr 14 03:18:25 user nova-compute[71323]: </domain>
Apr 14 03:18:25 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:18:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance destroyed successfully.
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] No VIF found with MAC fa:16:3e:02:a9:64, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.345s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:18:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.policy [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c1390cde350341c3ac0275e429146cf8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b689b65111045168cf21c51bf1274f2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:18:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Creating image(s)
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "/opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk 1073741824" returned: 0 in 0.061s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.197s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking if we can resize image /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Cannot resize image /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'migration_context' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Ensure instance console log exists: /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Successfully created port: 4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:27 user nova-compute[71323]: WARNING nova.compute.manager [req-e56428b7-196e-4f54-9b31-fd790bce1cbd req-e4170257-6e98-4e6b-80ac-2176b18d4e01 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state active and task_state rescuing.
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Successfully updated port: 4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquired lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-changed-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Refreshing instance network info cache due to event network-changed-4ceedbf8-a65d-4a77-b13c-837ae6290ea1. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] Acquiring lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating instance_info_cache with network_info: [{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Releasing lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Instance network_info: |[{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] Acquired lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.neutron [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Refreshing network info cache for port 4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Start _get_guest_xml network_info=[{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:18:27 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:27 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-260481439',display_name='tempest-AttachVolumeNegativeTest-server-260481439',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-260481439',id=38,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP/ufTFDXnGMjwB2FSg+A9mMX0vCesLJiAzD3NMUcRVyCg0qAk7buF7y/zaESmh9YTRgnJRFZQSsoliYulIJhfUVJEsLn5+ezAqOMz+G6LfzgivXohWdPYf/eGFCH+h13A==',key_name='tempest-keypair-1704381483',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-mhfzow8a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:18:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=04737acf-2aa7-4f8c-a33d-fa2b50ea29d7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.objects.instance [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'pci_devices' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:18:27 user nova-compute[71323]:   <uuid>04737acf-2aa7-4f8c-a33d-fa2b50ea29d7</uuid>
Apr 14 03:18:27 user nova-compute[71323]:   <name>instance-00000026</name>
Apr 14 03:18:27 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:18:27 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:18:27 user nova-compute[71323]:   <metadata>
Apr 14 03:18:27 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:18:27 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeNegativeTest-server-260481439</nova:name>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:18:27</nova:creationTime>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:18:27 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:18:27 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:owner>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:user uuid="c1390cde350341c3ac0275e429146cf8">tempest-AttachVolumeNegativeTest-376301777-project-member</nova:user>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:project uuid="3b689b65111045168cf21c51bf1274f2">tempest-AttachVolumeNegativeTest-376301777</nova:project>
Apr 14 03:18:27 user nova-compute[71323]:       </nova:owner>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:18:27 user nova-compute[71323]:       <nova:ports>
Apr 14 03:18:27 user nova-compute[71323]:         <nova:port uuid="4ceedbf8-a65d-4a77-b13c-837ae6290ea1">
Apr 14 03:18:27 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 03:18:27 user nova-compute[71323]:         </nova:port>
Apr 14 03:18:27 user nova-compute[71323]:       </nova:ports>
Apr 14 03:18:27 user nova-compute[71323]:     </nova:instance>
Apr 14 03:18:27 user nova-compute[71323]:   </metadata>
Apr 14 03:18:27 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:18:27 user nova-compute[71323]:     <system>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="serial">04737acf-2aa7-4f8c-a33d-fa2b50ea29d7</entry>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="uuid">04737acf-2aa7-4f8c-a33d-fa2b50ea29d7</entry>
Apr 14 03:18:27 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:18:27 user nova-compute[71323]:     </system>
Apr 14 03:18:27 user nova-compute[71323]:   </sysinfo>
Apr 14 03:18:27 user nova-compute[71323]:   <os>
Apr 14 03:18:27 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:18:27 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:18:27 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:18:27 user nova-compute[71323]:   </os>
Apr 14 03:18:27 user nova-compute[71323]:   <features>
Apr 14 03:18:27 user nova-compute[71323]:     <acpi/>
Apr 14 03:18:27 user nova-compute[71323]:     <apic/>
Apr 14 03:18:27 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:18:27 user nova-compute[71323]:   </features>
Apr 14 03:18:27 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:18:27 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:18:27 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:18:27 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:18:27 user nova-compute[71323]:   </clock>
Apr 14 03:18:27 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:18:27 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:18:27 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:18:27 user nova-compute[71323]:   </cpu>
Apr 14 03:18:27 user nova-compute[71323]:   <devices>
Apr 14 03:18:27 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:18:27 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:18:27 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk"/>
Apr 14 03:18:27 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:18:27 user nova-compute[71323]:     </disk>
Apr 14 03:18:27 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:18:27 user nova-compute[71323]:       <mac address="fa:16:3e:ce:8b:f2"/>
Apr 14 03:18:27 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:18:27 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:18:27 user nova-compute[71323]:       <target dev="tap4ceedbf8-a6"/>
Apr 14 03:18:27 user nova-compute[71323]:     </interface>
Apr 14 03:18:27 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:18:27 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/console.log" append="off"/>
Apr 14 03:18:27 user nova-compute[71323]:     </serial>
Apr 14 03:18:27 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:18:27 user nova-compute[71323]:     <video>
Apr 14 03:18:27 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:18:27 user nova-compute[71323]:     </video>
Apr 14 03:18:27 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:18:27 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:18:27 user nova-compute[71323]:     </rng>
Apr 14 03:18:27 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:18:27 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:18:27 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:18:27 user nova-compute[71323]:     </memballoon>
Apr 14 03:18:27 user nova-compute[71323]:   </devices>
Apr 14 03:18:27 user nova-compute[71323]: </domain>
Apr 14 03:18:27 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-260481439',display_name='tempest-AttachVolumeNegativeTest-server-260481439',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-260481439',id=38,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP/ufTFDXnGMjwB2FSg+A9mMX0vCesLJiAzD3NMUcRVyCg0qAk7buF7y/zaESmh9YTRgnJRFZQSsoliYulIJhfUVJEsLn5+ezAqOMz+G6LfzgivXohWdPYf/eGFCH+h13A==',key_name='tempest-keypair-1704381483',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-mhfzow8a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:18:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=04737acf-2aa7-4f8c-a33d-fa2b50ea29d7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG os_vif [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap4ceedbf8-a6, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap4ceedbf8-a6, col_values=(('external_ids', {'iface-id': '4ceedbf8-a65d-4a77-b13c-837ae6290ea1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ce:8b:f2', 'vm-uuid': '04737acf-2aa7-4f8c-a33d-fa2b50ea29d7'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:18:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:27 user nova-compute[71323]: INFO os_vif [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6')
Apr 14 03:18:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:18:28 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:ce:8b:f2, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:18:28 user nova-compute[71323]: DEBUG nova.network.neutron [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updated VIF entry in instance network info cache for port 4ceedbf8-a65d-4a77-b13c-837ae6290ea1. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:18:28 user nova-compute[71323]: DEBUG nova.network.neutron [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating instance_info_cache with network_info: [{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:18:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-0ffa701f-652d-494d-a83b-49252913f408 req-80ac9b16-1aab-4e4a-b034-0fe1af054e7f service nova] Releasing lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for ca85d0c8-d192-4bdf-b607-6fe637ad6f81 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442309.1251018, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:18:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Resumed (Lifecycle Event)
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-809fedca-0e6c-44c6-a6a7-59a587a82108 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:18:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] During sync_power_state the instance has a pending task (rescuing). Skip.
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442309.138785, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:18:29 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Started (Lifecycle Event)
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:29 user nova-compute[71323]: WARNING nova.compute.manager [req-cc3d62f0-5c55-44c7-87b0-6b4277835476 req-de7d9c1e-063b-441e-a5a7-360501885219 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state rescued and task_state None.
Apr 14 03:18:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:29 user nova-compute[71323]: DEBUG nova.compute.manager [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] No waiting events found dispatching network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:29 user nova-compute[71323]: WARNING nova.compute.manager [req-3f82a5cb-0867-45ab-8311-9527fe2fa85c req-297f1f29-dda5-4c03-8eda-dadee4980080 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received unexpected event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 for instance with vm_state building and task_state spawning.
Apr 14 03:18:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442311.7499828, 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] VM Resumed (Lifecycle Event)
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Instance spawned successfully.
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442311.7528749, 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] VM Started (Lifecycle Event)
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] No waiting events found dispatching network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:18:31 user nova-compute[71323]: WARNING nova.compute.manager [req-76a3843a-f41d-4422-8fb4-26c8233b5b52 req-65decf19-127a-4c7b-bfbe-4907e97b06e9 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received unexpected event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 for instance with vm_state building and task_state spawning.
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Took 5.88 seconds to spawn the instance on the hypervisor.
Apr 14 03:18:31 user nova-compute[71323]: DEBUG nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:18:31 user nova-compute[71323]: INFO nova.compute.manager [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Took 6.55 seconds to build instance.
Apr 14 03:18:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-aceced2e-7dfc-4ff6-9fc3-0d43cc25deb5 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.658s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:18:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:18:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:00 user nova-compute[71323]: INFO nova.compute.manager [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attaching volume 64d3b548-9384-41f0-b6cb-e80306d79286 to /dev/vdb
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.utils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b2f11ae-dc29-4540-8508-e3cb7a54bb2c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[35f1686b-a05f-4988-9d13-0ab94d295dfb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fe2d664-3e07-4fba-afe7-322f1b28b230]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[84c3a993-2345-426a-969e-8757d2b5b093]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:19:00 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:19:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a44a9c9d-bc62-4880-8bea-077168d3fa70]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG os_brick.utils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:19:00 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating existing volume attachment record: 0e9fa941-9e01-480d-91e5-6cbd3953fca1 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:19:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YzwifkRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:19:04 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:19:04 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:19:04 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8393add3-1984-49d9-ac59-3a6e34177128]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:19:04 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d54ba331-d216-4130-92de-b64799a1f2f3]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286] added
Apr 14 03:19:04 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd47f57f-7ad1-47fb-9c10-a4e265a95ebf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfd48767-0a89-465b-bcbb-efc84b4746d7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YzwifkRE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YzwifkRE" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bb9ffed-7d7c-40a8-ad15-e294b25e9d3a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YzwifkRE'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7dcbafae-4bf0-41be-9b8f-87c57a7fea38]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[afa8630a-b63e-46db-8c93-d9af6053dedd]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b956b87-2897-4962-a086-4631b2ea1ddc]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260]
Apr 14 03:19:05 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260] successful.
Apr 14 03:19:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[08752ae1-2b17-4051-8469-7e0e021b7f45]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c071ca1-7416-4345-9758-4bb5496c4609]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)
Apr 14 03:19:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:05 user nova-compute[71323]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)
Apr 14 03:19:05 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.320s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] HCTL ('35', '-', '-', 0) found on session 45 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[189d3b9a-043e-4b25-a0cb-b5c5dfd8ccac]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:05 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 45 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Searching for a device in session 45 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YzwifkRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.359s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== connect_volume: return (1363ms) {'type': 'block', 'scsi_wwn': '26133356332366330', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26133356332366330', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:19:06 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:19:06 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:19:06 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:19:06 user nova-compute[71323]:   <serial>64d3b548-9384-41f0-b6cb-e80306d79286</serial>
Apr 14 03:19:06 user nova-compute[71323]: </disk>
Apr 14 03:19:06 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] No VIF found with MAC fa:16:3e:dd:ec:3f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:19:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6c2e3417-bbaf-40cb-8890-ea87d0461c84 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.481s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:07 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:19:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:08 user nova-compute[71323]: INFO nova.compute.manager [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Detaching volume 64d3b548-9384-41f0-b6cb-e80306d79286
Apr 14 03:19:08 user nova-compute[71323]: INFO nova.virt.block_device [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Attempting to driver detach volume 64d3b548-9384-41f0-b6cb-e80306d79286 from mountpoint /dev/vdb
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Attempting to detach device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:19:08 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:19:08 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:19:08 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:19:08 user nova-compute[71323]:   <serial>64d3b548-9384-41f0-b6cb-e80306d79286</serial>
Apr 14 03:19:08 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:19:08 user nova-compute[71323]: </disk>
Apr 14 03:19:08 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:19:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the persistent domain config.
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:19:08 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:19:08 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:19:08 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:19:08 user nova-compute[71323]:   <serial>64d3b548-9384-41f0-b6cb-e80306d79286</serial>
Apr 14 03:19:08 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:19:08 user nova-compute[71323]: </disk>
Apr 14 03:19:08 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442348.8763745, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:19:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully detached device vdb from instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 from the live domain config.
Apr 14 03:19:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YzwifkRE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:19:08 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9600fdc8-03af-4d72-9b59-cd7d7e4481f0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:19:08 user nova-compute[71323]: No targets found.
Apr 14 03:19:08 user nova-compute[71323]: iSNS:
Apr 14 03:19:08 user nova-compute[71323]: No targets found.
Apr 14 03:19:08 user nova-compute[71323]: STATIC:
Apr 14 03:19:08 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286
Apr 14 03:19:08 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:19:08 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:19:08 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:19:08 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:19:08 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:19:08 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:19:08 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:19:08 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:19:08 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:19:08 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:19:08 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:19:08 user nova-compute[71323]: FIRMWARE:
Apr 14 03:19:08 user nova-compute[71323]: No targets found.
Apr 14 03:19:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:19:08 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:19:08 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[caf7ada8-6dbf-445d-895e-32957b87583b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e70ae468-faa1-4aa8-bdcf-faf335ad80f1]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)
Apr 14 03:19:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:08 user nova-compute[71323]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 (non-flash)
Apr 14 03:19:08 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1a394d30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e406037d-6ee4-4496-a080-c5544596e45d]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaf57d2b-7714-4f65-bab2-37c08c5e8933]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[106ff6af-42c6-4eb8-9432-ea915f10cec7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.044s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[22cfdecb-59bb-4a7a-ba3f-b94909032b88]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7611f026-4837-4517-9028-5637d7555d5e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9591b1d-e389-456b-bc4e-222755682c54]: (4, ('Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260]\nLogout of [sid: 45, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260]
Apr 14 03:19:09 user nova-compute[71323]: Logout of [sid: 45, target: iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286, portal: 172.16.0.220,3260] successful.
Apr 14 03:19:09 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:64d3b548-9384-41f0-b6cb-e80306d79286 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[261135e4-a998-4512-9a98-73ea7a5ec210]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.306s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] <== disconnect_volume: return (307ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'flavor' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6bf1cc2b-8bdd-43d8-a61d-d6e876840d85 tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.605s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:19:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.159s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:19:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7882MB free_disk=26.180992126464844GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8cabfa40-3167-4f13-b6f3-4064d326f837 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.475s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:19:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:19:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:19:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:19:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:19:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.064s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:40 user nova-compute[71323]: INFO nova.compute.manager [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Attaching volume 96f437f9-4897-417a-95e3-097d2ff29476 to /dev/vdb
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.utils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[58b5f83f-0deb-43f6-a7f9-e8c5f6d9f1ee]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2efe6af6-3e5d-49fb-bac0-c583d33a8090]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b8784f6-3111-4532-a948-cc696e77030a]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[20e151f4-5d92-4413-b740-1cde0e0444ae]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:19:40 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:19:40 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a621015-6c5f-442f-8b6f-4375d84efed0]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG os_brick.utils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== get_connector_properties: return (130ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:19:40 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating existing volume attachment record: bca5b8c8-48bf-4f16-a19b-3e953f04cdc1 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:19:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:19:44 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:19:44 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:19:44 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a959b54a-eab3-41ef-9320-a35898feaa01]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:19:44 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a031f1e-9cc5-411e-9948-3aeb6f8ff4e0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476] added
Apr 14 03:19:44 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a890dba-23d9-4055-b4a6-322fa89b5cec]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8649d139-0912-41ca-9955-79db396720dc]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JTPOdpZX {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JTPOdpZX" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f0ba813-7159-4fcf-b642-1f095e4892a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'JTPOdpZX'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e072873-ab6f-4067-bd58-43a4cd8ef4e6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3659d826-557e-4b0d-8ec3-c358ac87b0f7]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[69d08f3d-0ad4-4693-ae95-c5131786509f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]
Apr 14 03:19:45 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.
Apr 14 03:19:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b685f8d6-a34f-462e-9613-0608c96b50da]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca9eff77-462a-4470-93c8-49e2928b318c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:19:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:19:45 user nova-compute[71323]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:19:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.293s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] HCTL ('35', '-', '-', 0) found on session 46 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[49eef9ca-3260-44b9-8afd-92c990a04683]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Searching for a device in session 46 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:19:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.327s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== connect_volume: return (1328ms) {'type': 'block', 'scsi_wwn': '23737626134303438', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23737626134303438', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:19:46 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:19:46 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:19:46 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:19:46 user nova-compute[71323]:   <serial>96f437f9-4897-417a-95e3-097d2ff29476</serial>
Apr 14 03:19:46 user nova-compute[71323]: </disk>
Apr 14 03:19:46 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No VIF found with MAC fa:16:3e:71:a9:a9, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:19:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5c78ea36-935b-4ba8-bd6a-b9b75fee2035 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.561s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing instance network info cache due to event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:19:53 user nova-compute[71323]: INFO nova.compute.manager [None req-0753309c-6dc0-4e2b-b177-e0e339ef2f63 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Get console output
Apr 14 03:19:53 user nova-compute[71323]: INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
Apr 14 03:19:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updated VIF entry in instance network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:19:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d73f0ddf-43c5-443c-b846-01e6938f278f req-749fe423-cdd3-43b9-9bec-04deb41bf756 service nova] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:19:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing instance network info cache due to event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updated VIF entry in instance network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:19:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1d391c1-08cd-4f86-8960-944ebf31962c req-1ccc9893-6139-49bf-b802-86cfec0f95e1 service nova] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:19:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:58 user nova-compute[71323]: INFO nova.compute.manager [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Rebooting instance
Apr 14 03:19:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:19:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:19:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:19:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:19:59 user nova-compute[71323]: WARNING nova.compute.manager [req-42822657-10e4-4c51-adee-2f6b2424d118 req-2ad74ccf-10ec-4876-a17e-da2ea40ec835 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 03:20:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance destroyed successfully.
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'resources' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:17:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:19:59Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa478a9ea-0f, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:00 user nova-compute[71323]: INFO os_vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f')
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:20:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[308db558-9cb8-4c06-b6bb-071ef5772b08]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:20:00 user nova-compute[71323]: No targets found.
Apr 14 03:20:00 user nova-compute[71323]: iSNS:
Apr 14 03:20:00 user nova-compute[71323]: No targets found.
Apr 14 03:20:00 user nova-compute[71323]: STATIC:
Apr 14 03:20:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:20:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476
Apr 14 03:20:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:20:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:00 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:20:00 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:00 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:00 user nova-compute[71323]: FIRMWARE:
Apr 14 03:20:00 user nova-compute[71323]: No targets found.
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:20:00 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:20:00 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bc98ffd-6cb5-43ef-8ad2-d0a21fe34b41]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed6a689b-d959-4cab-b4b7-78d13dfd2fcd]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1a37fbe0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[673831c5-8ffd-4745-b1e0-bd4a2056a67c]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7def4894-aef6-491f-a95f-21e2a07022f1]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d04578bc-a08d-406e-bab2-8a1a8c6c440c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f07940a-0c7a-4dbf-9025-5345f8a67819]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d5454a0-6c7a-4e8f-8524-341cef4ace90]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d61a8aa5-88e7-4534-b6bf-4e65c38d41fb]: (4, ('Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]\nLogout of [sid: 46, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]
Apr 14 03:20:00 user nova-compute[71323]: Logout of [sid: 46, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[40d96463-ddce-4b29-91a5-9ca5e53f2049]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.309s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== disconnect_volume: return (310ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start _get_guest_xml network_info=[{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=3af87ed2-d2c6-4608-b72f-8b0f91e4ca86,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '3af87ed2-d2c6-4608-b72f-8b0f91e4ca86'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': 'bca5b8c8-48bf-4f16-a19b-3e953f04cdc1', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, 'status': 'reserved', 'instance': '8cabfa40-3167-4f13-b6f3-4064d326f837', 'attached_at': '', 'detached_at': '', 'volume_id': '96f437f9-4897-417a-95e3-097d2ff29476', 'serial': '96f437f9-4897-417a-95e3-097d2ff29476'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdb', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:20:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:00 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=3af87ed2-d2c6-4608-b72f-8b0f91e4ca86,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'vcpu_model' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:20:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:00 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:00 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d36e59d-9112-4241-bb1e-3a62c967f1d5]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:20:00 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0f15e96-ed48-4415-a58b-9f50111ba9b0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476] added
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[092b7164-72f8-4c58-80d1-d00e53fbc394]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5456389a-f44f-483b-8ebf-1c42f0d811b7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JTPOdpZX {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JTPOdpZX" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[19b04b20-8e46-4704-8768-d144370ff1b5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'JTPOdpZX'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd109fb1-0293-4b2f-946a-9cb832f26a7b]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8969d187-4481-4691-9bbd-ede4fbec651c]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a434c74f-d660-49fc-adb9-41395a93ce55]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]
Apr 14 03:20:00 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e910125-a211-41ae-ab9e-6e071d8f3a16]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8101ae32-c148-43b9-a907-9c41716e2815]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:00 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:00 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.303s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] HCTL ('35', '-', '-', 0) found on session 47 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[49a5a139-01eb-4b08-ab0b-ba18113fffe1]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:00 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Searching for a device in session 47 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:01 user nova-compute[71323]: WARNING nova.compute.manager [req-8faf76ca-5d72-40c4-ad30-84205a9f31a9 req-0b88d6a5-ee53-4ca5-ae28-dd54fe6a981e service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 03:20:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Searching for a device in session 47 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.337s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== connect_volume: return (1338ms) {'type': 'block', 'scsi_wwn': '23737626134303438', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23737626134303438', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:17:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:19:59Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.objects.instance [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'pci_devices' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:20:01 user nova-compute[71323]:   <uuid>8cabfa40-3167-4f13-b6f3-4064d326f837</uuid>
Apr 14 03:20:01 user nova-compute[71323]:   <name>instance-00000025</name>
Apr 14 03:20:01 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:20:01 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:20:01 user nova-compute[71323]:   <metadata>
Apr 14 03:20:01 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:20:01 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:name>tempest-TestMinimumBasicScenario-server-252373748</nova:name>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:20:00</nova:creationTime>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:20:01 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:20:01 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:owner>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:user uuid="12199a43298942898a2e694ee2c5230e">tempest-TestMinimumBasicScenario-144187586-project-member</nova:user>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:project uuid="0bd451b3f6734fe3ac0a8e8a853bb9eb">tempest-TestMinimumBasicScenario-144187586</nova:project>
Apr 14 03:20:01 user nova-compute[71323]:       </nova:owner>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:root type="image" uuid="3af87ed2-d2c6-4608-b72f-8b0f91e4ca86"/>
Apr 14 03:20:01 user nova-compute[71323]:       <nova:ports>
Apr 14 03:20:01 user nova-compute[71323]:         <nova:port uuid="a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0">
Apr 14 03:20:01 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 03:20:01 user nova-compute[71323]:         </nova:port>
Apr 14 03:20:01 user nova-compute[71323]:       </nova:ports>
Apr 14 03:20:01 user nova-compute[71323]:     </nova:instance>
Apr 14 03:20:01 user nova-compute[71323]:   </metadata>
Apr 14 03:20:01 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:20:01 user nova-compute[71323]:     <system>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="serial">8cabfa40-3167-4f13-b6f3-4064d326f837</entry>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="uuid">8cabfa40-3167-4f13-b6f3-4064d326f837</entry>
Apr 14 03:20:01 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:20:01 user nova-compute[71323]:     </system>
Apr 14 03:20:01 user nova-compute[71323]:   </sysinfo>
Apr 14 03:20:01 user nova-compute[71323]:   <os>
Apr 14 03:20:01 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:20:01 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:20:01 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:20:01 user nova-compute[71323]:   </os>
Apr 14 03:20:01 user nova-compute[71323]:   <features>
Apr 14 03:20:01 user nova-compute[71323]:     <acpi/>
Apr 14 03:20:01 user nova-compute[71323]:     <apic/>
Apr 14 03:20:01 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:20:01 user nova-compute[71323]:   </features>
Apr 14 03:20:01 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:20:01 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:20:01 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:20:01 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:20:01 user nova-compute[71323]:   </clock>
Apr 14 03:20:01 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:20:01 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:20:01 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:20:01 user nova-compute[71323]:   </cpu>
Apr 14 03:20:01 user nova-compute[71323]:   <devices>
Apr 14 03:20:01 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:20:01 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:20:01 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk"/>
Apr 14 03:20:01 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:20:01 user nova-compute[71323]:     </disk>
Apr 14 03:20:01 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:20:01 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:20:01 user nova-compute[71323]:       <source dev="/dev/sdd"/>
Apr 14 03:20:01 user nova-compute[71323]:       <target dev="vdb" bus="virtio"/>
Apr 14 03:20:01 user nova-compute[71323]:       <serial>96f437f9-4897-417a-95e3-097d2ff29476</serial>
Apr 14 03:20:01 user nova-compute[71323]:     </disk>
Apr 14 03:20:01 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:20:01 user nova-compute[71323]:       <mac address="fa:16:3e:71:a9:a9"/>
Apr 14 03:20:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:20:01 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:20:01 user nova-compute[71323]:       <target dev="tapa478a9ea-0f"/>
Apr 14 03:20:01 user nova-compute[71323]:     </interface>
Apr 14 03:20:01 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:20:01 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/console.log" append="off"/>
Apr 14 03:20:01 user nova-compute[71323]:     </serial>
Apr 14 03:20:01 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:20:01 user nova-compute[71323]:     <video>
Apr 14 03:20:01 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:20:01 user nova-compute[71323]:     </video>
Apr 14 03:20:01 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:20:01 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:20:01 user nova-compute[71323]:     </rng>
Apr 14 03:20:01 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:20:01 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:20:01 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:20:01 user nova-compute[71323]:     </memballoon>
Apr 14 03:20:01 user nova-compute[71323]:   </devices>
Apr 14 03:20:01 user nova-compute[71323]: </domain>
Apr 14 03:20:01 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:20:01 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'trusted_certs' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Cannot resize image /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'migration_context' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:17:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:19:59Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG os_vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa478a9ea-0f, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa478a9ea-0f, col_values=(('external_ids', {'iface-id': 'a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:71:a9:a9', 'vm-uuid': '8cabfa40-3167-4f13-b6f3-4064d326f837'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:20:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:02 user nova-compute[71323]: INFO os_vif [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f')
Apr 14 03:20:03 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:04 user nova-compute[71323]: WARNING nova.compute.manager [req-f823d95c-99bd-4bc7-9bec-962485566d56 req-b52edbaa-aee4-4fa4-bc94-38764df258dc service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 03:20:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for 8cabfa40-3167-4f13-b6f3-4064d326f837 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442406.1562493, 8cabfa40-3167-4f13-b6f3-4064d326f837 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] VM Resumed (Lifecycle Event)
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:06 user nova-compute[71323]: WARNING nova.compute.manager [req-df0e6fc9-642c-4e8f-b848-f9121bf71d9e req-40785a21-27ef-44e6-bd52-ee55227e6dfd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state active and task_state reboot_started_hard.
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:06 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance rebooted successfully.
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] During sync_power_state the instance has a pending task (reboot_started_hard). Skip.
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442406.160492, 8cabfa40-3167-4f13-b6f3-4064d326f837 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] VM Started (Lifecycle Event)
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:06 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: reboot_started_hard, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:06 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] During sync_power_state the instance has a pending task (reboot_started_hard). Skip.
Apr 14 03:20:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-686a2134-fa40-4b25-a9cb-8492a4424ac8 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 7.582s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:20:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:20:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81/disk.rescue --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:11 user nova-compute[71323]: INFO nova.compute.manager [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Unrescuing
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquired lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating instance_info_cache with network_info: [{"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Releasing lock "refresh_cache-ca85d0c8-d192-4bdf-b607-6fe637ad6f81" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'flavor' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:13 user nova-compute[71323]: WARNING nova.compute.manager [req-9e0ed0c8-e735-441e-86e7-dbc1a0310c24 req-6a690e0f-3df6-42c5-b84c-1e47621c6993 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:20:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7973MB free_disk=26.17074203491211GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance destroyed successfully.
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 192, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8cabfa40-3167-4f13-b6f3-4064d326f837 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1728MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:20:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.608s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:15 user nova-compute[71323]: WARNING nova.compute.manager [req-c6404d0e-c6a5-4039-9d69-419ef13637ed req-9c54c5a6-c61c-4ac7-be45-32cb0501a9b1 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:20:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:15 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:20:16 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [{"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:20:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-changed-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Refreshing instance network info cache due to event network-changed-4ceedbf8-a65d-4a77-b13c-837ae6290ea1. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] Acquiring lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] Acquired lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Refreshing network info cache for port 4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:17 user nova-compute[71323]: WARNING nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:17 user nova-compute[71323]: WARNING nova.compute.manager [req-8e273f15-9f63-40ff-9418-f23c4727b62b req-2b02fad9-64fc-4e59-bca5-8ea10e99e81a service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state rescued and task_state unrescuing.
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for ca85d0c8-d192-4bdf-b607-6fe637ad6f81 due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442417.3688211, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Resumed (Lifecycle Event)
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f24c2e95-ed90-476b-bbd7-f5b728ed8314 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: rescued, current task_state: unrescuing, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] During sync_power_state the instance has a pending task (unrescuing). Skip.
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442417.37452, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:17 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Started (Lifecycle Event)
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: None, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updated VIF entry in instance network info cache for port 4ceedbf8-a65d-4a77-b13c-837ae6290ea1. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG nova.network.neutron [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating instance_info_cache with network_info: [{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-949a893e-8e74-4fa7-a195-bd77b5d6b4f1 req-332fda69-d818-49c8-9651-9f3843912036 service nova] Releasing lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.compute.manager [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing instance network info cache due to event network-changed-73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] Acquiring lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] Acquired lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.network.neutron [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Refreshing network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.network.neutron [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updated VIF entry in instance network info cache for port 73d42070-416a-497f-b3d4-8467d8255c5f. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.network.neutron [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating instance_info_cache with network_info: [{"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-629eb2ec-c263-474a-84b9-169207390c09 req-a8da617e-6618-4a9b-b320-90ba9732efb3 service nova] Releasing lock "refresh_cache-9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG nova.objects.instance [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:20 user nova-compute[71323]: INFO nova.compute.manager [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Attaching volume c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 to /dev/vdb
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[86a00c17-2517-432a-aa45-8b91b240cd88]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fab8c01-aea7-45de-8973-b468c0d87065]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7badc4ef-8ef7-463a-bc06-1143983d6efe]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[92623d21-fb3b-429e-8385-ab831378ca87]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:20:20 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:20:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f39d0ca-9381-4e16-ba48-7f2057ee8f42]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:20:20 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating existing volume attachment record: b2efd92e-831d-4fa4-b5ef-ca0092a85ed6 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:22 user nova-compute[71323]: INFO nova.compute.manager [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Terminating instance
Apr 14 03:20:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-unplugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] No waiting events found dispatching network-vif-unplugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aca232ed-a184-4845-9fe6-d437fcaa9f4b req-13f00284-c86b-473e-8ff8-be32378541fd service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-unplugged-73d42070-416a-497f-b3d4-8467d8255c5f for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Instance destroyed successfully.
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.objects.instance [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lazy-loading 'resources' on Instance uuid 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:40Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-79587855',display_name='tempest-SnapshotDataIntegrityTests-server-79587855',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-79587855',id=34,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN4ITzOqj8xEk6dTBZ8Ot32M8Esvy3C6MqRifOQoX4ksa7bmDWj6LGvCYEbIDUI0lXUxkiCpG51Z30GVyclYEqcHtnzy83lAYeIu0VXyXlLqttu//3qnh/G2175vPwLjXQ==',key_name='tempest-SnapshotDataIntegrityTests-1339724753',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:13:47Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0096d0892acf4d5f8a0fa290a18b451a',ramdisk_id='',reservation_id='r-pmhvgxq9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-450991924',owner_user_name='tempest-SnapshotDataIntegrityTests-450991924-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:13:48Z,user_data=None,user_id='ba8614e27c65428d92e4bf01969976e7',uuid=9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converting VIF {"id": "73d42070-416a-497f-b3d4-8467d8255c5f", "address": "fa:16:3e:dd:ec:3f", "network": {"id": "f30776a2-73d7-4689-8046-ab07cd500f82", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-520298661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0096d0892acf4d5f8a0fa290a18b451a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap73d42070-41", "ovs_interfaceid": "73d42070-416a-497f-b3d4-8467d8255c5f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG os_vif [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap73d42070-41, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:20:23 user nova-compute[71323]: INFO os_vif [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:ec:3f,bridge_name='br-int',has_traffic_filtering=True,id=73d42070-416a-497f-b3d4-8467d8255c5f,network=Network(f30776a2-73d7-4689-8046-ab07cd500f82),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap73d42070-41')
Apr 14 03:20:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Deleting instance files /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4_del
Apr 14 03:20:23 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Deletion of /opt/stack/data/nova/instances/9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4_del complete
Apr 14 03:20:23 user nova-compute[71323]: INFO nova.compute.manager [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 03:20:23 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:23 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Took 0.45 seconds to deallocate network for instance.
Apr 14 03:20:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.282s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:24 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Deleted allocations for instance 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4
Apr 14 03:20:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-70948966-d9d2-4cf8-bdd6-85cfab6637bf tempest-SnapshotDataIntegrityTests-450991924 tempest-SnapshotDataIntegrityTests-450991924-project-member] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.803s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] Acquiring lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] Lock "9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] No waiting events found dispatching network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:24 user nova-compute[71323]: WARNING nova.compute.manager [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received unexpected event network-vif-plugged-73d42070-416a-497f-b3d4-8467d8255c5f for instance with vm_state deleted and task_state None.
Apr 14 03:20:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-69ff2501-b6fb-4c90-83ce-5c9f227fa3d1 req-beb16a1e-de6e-4e49-a7ea-aa01cd4feba5 service nova] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Received event network-vif-deleted-73d42070-416a-497f-b3d4-8467d8255c5f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YIukGbUA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:20:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:25 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf5dee83-17d1-455c-87d8-7644d6b5a337]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:20:25 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c30a0b1b-e6e3-40b0-a207-01a5ab5bbe69]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36] added
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[07bc199c-38ec-4674-972b-14b50330c8a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cb34798-92d3-452d-a511-5cfa60e3f492]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YIukGbUA {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YIukGbUA" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ed32b4d-2ade-4754-8046-37b25b797b29]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YIukGbUA'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b896138-1d26-444f-9e23-28e8df7bd513]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f10335a-bf95-4c0f-add5-48c19cd2c688]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[12639489-6928-4ec8-9831-6925fd310aa0]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260]
Apr 14 03:20:25 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260] successful.
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[65b8273a-96d4-4f59-8a17-d04d0e749f49]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e086a88-b669-4eb4-b5cb-05d291c2a7c5]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:20:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] HCTL ('36', '-', '-', 0) found on session 48 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e05fe4b-765b-4779-8864-8ddfaa173f92]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 48 and hctl ['36', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 48 and hctl ['36', '*', '*', 0] yield: sde {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YIukGbUA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.347s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== connect_volume: return (1348ms) {'type': 'block', 'scsi_wwn': '23164613665323061', 'path': '/dev/sde'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23164613665323061', 'path': '/dev/sde'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:20:26 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:20:26 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:20:26 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:20:26 user nova-compute[71323]:   <serial>c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36</serial>
Apr 14 03:20:26 user nova-compute[71323]: </disk>
Apr 14 03:20:26 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:20:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:ce:8b:f2, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:20:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-04a612b3-d249-41fa-8f92-f4b1c2794400 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 7.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:20:30 user nova-compute[71323]: INFO nova.compute.claims [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Claim successful on node user
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.357s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:20:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.policy [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c1390cde350341c3ac0275e429146cf8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b689b65111045168cf21c51bf1274f2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:20:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Creating image(s)
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "/opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:30 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk 1073741824" returned: 0 in 0.051s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.195s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Successfully created port: 66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json" returned: 0 in 0.162s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Cannot resize image /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'migration_context' on Instance uuid 9b91538e-2a4b-453b-bfb3-5e569da0a3cd {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Ensure instance console log exists: /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Successfully updated port: 66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquired lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-changed-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Refreshing instance network info cache due to event network-changed-66cc3bdb-6829-4939-95c9-3549b7c39c65. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] Acquiring lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updating instance_info_cache with network_info: [{"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Releasing lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Instance network_info: |[{"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] Acquired lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.neutron [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Refreshing network info cache for port 66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Start _get_guest_xml network_info=[{"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:20:32 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:32 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:20:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-532403844',display_name='tempest-AttachVolumeNegativeTest-server-532403844',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-532403844',id=39,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHSdJeK0I2g3r6BVK5roXFoHNwGS9/lEYRIk7CP+MH/Kk1cBXHAGb76eRSMrsWN/Ae392cfnXS4PUZ3wHpnCdSrlNAu5O+oD1xGzhmbUo4CzIPnk94ndEYbTDEjXXGjkuQ==',key_name='tempest-keypair-2056266938',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-009yzo9p',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:20:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=9b91538e-2a4b-453b-bfb3-5e569da0a3cd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'pci_devices' on Instance uuid 9b91538e-2a4b-453b-bfb3-5e569da0a3cd {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:20:32 user nova-compute[71323]:   <uuid>9b91538e-2a4b-453b-bfb3-5e569da0a3cd</uuid>
Apr 14 03:20:32 user nova-compute[71323]:   <name>instance-00000027</name>
Apr 14 03:20:32 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:20:32 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:20:32 user nova-compute[71323]:   <metadata>
Apr 14 03:20:32 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:20:32 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeNegativeTest-server-532403844</nova:name>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:20:32</nova:creationTime>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:20:32 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:20:32 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:owner>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:user uuid="c1390cde350341c3ac0275e429146cf8">tempest-AttachVolumeNegativeTest-376301777-project-member</nova:user>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:project uuid="3b689b65111045168cf21c51bf1274f2">tempest-AttachVolumeNegativeTest-376301777</nova:project>
Apr 14 03:20:32 user nova-compute[71323]:       </nova:owner>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:20:32 user nova-compute[71323]:       <nova:ports>
Apr 14 03:20:32 user nova-compute[71323]:         <nova:port uuid="66cc3bdb-6829-4939-95c9-3549b7c39c65">
Apr 14 03:20:32 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:20:32 user nova-compute[71323]:         </nova:port>
Apr 14 03:20:32 user nova-compute[71323]:       </nova:ports>
Apr 14 03:20:32 user nova-compute[71323]:     </nova:instance>
Apr 14 03:20:32 user nova-compute[71323]:   </metadata>
Apr 14 03:20:32 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:20:32 user nova-compute[71323]:     <system>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="serial">9b91538e-2a4b-453b-bfb3-5e569da0a3cd</entry>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="uuid">9b91538e-2a4b-453b-bfb3-5e569da0a3cd</entry>
Apr 14 03:20:32 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:20:32 user nova-compute[71323]:     </system>
Apr 14 03:20:32 user nova-compute[71323]:   </sysinfo>
Apr 14 03:20:32 user nova-compute[71323]:   <os>
Apr 14 03:20:32 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:20:32 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:20:32 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:20:32 user nova-compute[71323]:   </os>
Apr 14 03:20:32 user nova-compute[71323]:   <features>
Apr 14 03:20:32 user nova-compute[71323]:     <acpi/>
Apr 14 03:20:32 user nova-compute[71323]:     <apic/>
Apr 14 03:20:32 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:20:32 user nova-compute[71323]:   </features>
Apr 14 03:20:32 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:20:32 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:20:32 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:20:32 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:20:32 user nova-compute[71323]:   </clock>
Apr 14 03:20:32 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:20:32 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:20:32 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:20:32 user nova-compute[71323]:   </cpu>
Apr 14 03:20:32 user nova-compute[71323]:   <devices>
Apr 14 03:20:32 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:20:32 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:20:32 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk"/>
Apr 14 03:20:32 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:20:32 user nova-compute[71323]:     </disk>
Apr 14 03:20:32 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:20:32 user nova-compute[71323]:       <mac address="fa:16:3e:98:ec:25"/>
Apr 14 03:20:32 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:20:32 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:20:32 user nova-compute[71323]:       <target dev="tap66cc3bdb-68"/>
Apr 14 03:20:32 user nova-compute[71323]:     </interface>
Apr 14 03:20:32 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:20:32 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/console.log" append="off"/>
Apr 14 03:20:32 user nova-compute[71323]:     </serial>
Apr 14 03:20:32 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:20:32 user nova-compute[71323]:     <video>
Apr 14 03:20:32 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:20:32 user nova-compute[71323]:     </video>
Apr 14 03:20:32 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:20:32 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:20:32 user nova-compute[71323]:     </rng>
Apr 14 03:20:32 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:20:32 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:20:32 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:20:32 user nova-compute[71323]:     </memballoon>
Apr 14 03:20:32 user nova-compute[71323]:   </devices>
Apr 14 03:20:32 user nova-compute[71323]: </domain>
Apr 14 03:20:32 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:20:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-532403844',display_name='tempest-AttachVolumeNegativeTest-server-532403844',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-532403844',id=39,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHSdJeK0I2g3r6BVK5roXFoHNwGS9/lEYRIk7CP+MH/Kk1cBXHAGb76eRSMrsWN/Ae392cfnXS4PUZ3wHpnCdSrlNAu5O+oD1xGzhmbUo4CzIPnk94ndEYbTDEjXXGjkuQ==',key_name='tempest-keypair-2056266938',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-009yzo9p',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:20:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=9b91538e-2a4b-453b-bfb3-5e569da0a3cd,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG os_vif [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap66cc3bdb-68, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap66cc3bdb-68, col_values=(('external_ids', {'iface-id': '66cc3bdb-6829-4939-95c9-3549b7c39c65', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:98:ec:25', 'vm-uuid': '9b91538e-2a4b-453b-bfb3-5e569da0a3cd'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:32 user nova-compute[71323]: INFO os_vif [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68')
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:20:32 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:98:ec:25, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:20:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updated VIF entry in instance network info cache for port 66cc3bdb-6829-4939-95c9-3549b7c39c65. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:20:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updating instance_info_cache with network_info: [{"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e82d9602-5ea3-4bdc-939c-f5ad815cde57 req-cfceb10e-4f4c-4c84-8a01-a67993b49af1 service nova] Releasing lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] No waiting events found dispatching network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:34 user nova-compute[71323]: WARNING nova.compute.manager [req-38004762-0568-49e7-a37d-cfd0c57d7142 req-ffef1f7b-5f48-4d3d-b7bd-e6b4adb7f478 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received unexpected event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 for instance with vm_state building and task_state spawning.
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] No waiting events found dispatching network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:36 user nova-compute[71323]: WARNING nova.compute.manager [req-c95b7e36-58be-4055-983a-5a4e1dcfef99 req-02741a9b-c921-45b1-82fe-0bc0d7529701 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received unexpected event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 for instance with vm_state building and task_state spawning.
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442437.0198119, 9b91538e-2a4b-453b-bfb3-5e569da0a3cd => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] VM Resumed (Lifecycle Event)
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Instance spawned successfully.
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442437.0212345, 9b91538e-2a4b-453b-bfb3-5e569da0a3cd => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] VM Started (Lifecycle Event)
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Took 6.23 seconds to spawn the instance on the hypervisor.
Apr 14 03:20:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:37 user nova-compute[71323]: INFO nova.compute.manager [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Took 6.89 seconds to build instance.
Apr 14 03:20:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7ed10e4e-39ef-424e-96e1-25d312ddc1e9 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.985s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:38 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442423.2238917, 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:20:38 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] VM Stopped (Lifecycle Event)
Apr 14 03:20:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-369d9565-decf-4468-b85e-46b85c0e5562 None None] [instance: 9816a2bd-cc75-48ed-a4ce-6c5d03ed61e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:20:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:53 user nova-compute[71323]: INFO nova.compute.manager [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Terminating instance
Apr 14 03:20:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-d60d3f21-27a1-45c1-ad2d-e4f3b73568e4 req-9b1dec06-277b-4d5a-b486-c881114be5b7 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-unplugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Instance destroyed successfully.
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.objects.instance [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid bc7bf9fb-977a-4015-8739-6fa66c1ec210 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:14:18Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-463419112',display_name='tempest-ServerActionsTestJSON-server-463419112',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(12),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-463419112',id=35,image_ref='',info_cache=InstanceInfoCache,instance_type_id=12,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:16:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=192,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-pxncq0ij',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:18:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=bc7bf9fb-977a-4015-8739-6fa66c1ec210,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "address": "fa:16:3e:b1:de:25", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf1dbc6da-e8", "ovs_interfaceid": "f1dbc6da-e8f9-4962-ac4c-db69105689f9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_vif [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf1dbc6da-e8, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:20:54 user nova-compute[71323]: INFO os_vif [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:b1:de:25,bridge_name='br-int',has_traffic_filtering=True,id=f1dbc6da-e8f9-4962-ac4c-db69105689f9,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf1dbc6da-e8')
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'tUxPOhHN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:20:54 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[604aa1b1-0726-4bd3-abc0-6d0a2fd56801]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:20:54 user nova-compute[71323]: No targets found.
Apr 14 03:20:54 user nova-compute[71323]: iSNS:
Apr 14 03:20:54 user nova-compute[71323]: No targets found.
Apr 14 03:20:54 user nova-compute[71323]: STATIC:
Apr 14 03:20:54 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:20:54 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:54 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:54 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476
Apr 14 03:20:54 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:54 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:54 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:20:54 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:54 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:54 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36
Apr 14 03:20:54 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:54 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:54 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab
Apr 14 03:20:54 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:20:54 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:20:54 user nova-compute[71323]: FIRMWARE:
Apr 14 03:20:54 user nova-compute[71323]: No targets found.
Apr 14 03:20:54 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:20:54 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:20:54 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[440c454b-c424-428f-baed-7160446694cf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0aff133e-1cde-4fc3-aca2-73bb3b1be8f8]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:20:54 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:20:54 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:20:54 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b002432e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d2df483-24d4-4be8-98d9-b738b476709b]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[98b23552-fca6-483c-be43-9a911cbad704]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[49c4c38a-ef4a-43ec-8872-0c68e6502736]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.034s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b539c0e7-1993-422b-bfe2-1451a0f0e04f]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23031383739646233', '/dev/disk/by-id/scsi-1SCST_BIOe41848c-01879db36f93b921', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_e41848c') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[429d18e3-065a-462e-9d79-e3a1859c8f2f]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[399b108a-abed-44c1-b5c3-7f1025ceeb1c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ac768ee-a36d-423d-bbcf-4de561818dd0]: (4, ('Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]\nLogout of [sid: 40, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260]
Apr 14 03:20:54 user nova-compute[71323]: Logout of [sid: 40, target: iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab, portal: 172.16.0.220,3260] successful.
Apr 14 03:20:54 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7378e388-0e78-4212-b0ce-a1e082ba31ab -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18e37f56-7ba2-4fb3-82e5-5c6046326727]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] <== disconnect_volume: return (308ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:20:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Deleting instance files /opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210_del
Apr 14 03:20:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Deletion of /opt/stack/data/nova/instances/bc7bf9fb-977a-4015-8739-6fa66c1ec210_del complete
Apr 14 03:20:54 user nova-compute[71323]: INFO nova.compute.manager [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Took 1.14 seconds to destroy the instance on the hypervisor.
Apr 14 03:20:54 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:20:54 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:20:55 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:55 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Took 0.73 seconds to deallocate network for instance.
Apr 14 03:20:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ebcaa12-0f44-4e41-b88a-367a9a93c543 req-08503f7f-b586-4f8a-a2d2-a7705895f337 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-deleted-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:55 user nova-compute[71323]: INFO nova.compute.manager [req-0ebcaa12-0f44-4e41-b88a-367a9a93c543 req-08503f7f-b586-4f8a-a2d2-a7705895f337 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Neutron deleted interface f1dbc6da-e8f9-4962-ac4c-db69105689f9; detaching it from the instance and deleting it from the info cache
Apr 14 03:20:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-0ebcaa12-0f44-4e41-b88a-367a9a93c543 req-08503f7f-b586-4f8a-a2d2-a7705895f337 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:20:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-0ebcaa12-0f44-4e41-b88a-367a9a93c543 req-08503f7f-b586-4f8a-a2d2-a7705895f337 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Detach interface failed, port_id=f1dbc6da-e8f9-4962-ac4c-db69105689f9, reason: Instance bc7bf9fb-977a-4015-8739-6fa66c1ec210 could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] Acquiring lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] No waiting events found dispatching network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:20:56 user nova-compute[71323]: WARNING nova.compute.manager [req-e257177a-b19f-4795-94a9-bf2567ab9004 req-35aa9239-ba5b-4709-b732-5fd991fe2465 service nova] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Received unexpected event network-vif-plugged-f1dbc6da-e8f9-4962-ac4c-db69105689f9 for instance with vm_state active and task_state deleting.
Apr 14 03:20:56 user nova-compute[71323]: INFO nova.compute.manager [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Took 0.82 seconds to detach 1 volumes for instance.
Apr 14 03:20:56 user nova-compute[71323]: DEBUG nova.compute.manager [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Deleting volume: 7378e388-0e78-4212-b0ce-a1e082ba31ab {{(pid=71323) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:20:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.283s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:56 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Deleted allocations for instance bc7bf9fb-977a-4015-8739-6fa66c1ec210
Apr 14 03:20:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-2db03106-21d6-4b86-959b-e99df1516856 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "bc7bf9fb-977a-4015-8739-6fa66c1ec210" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.237s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:20:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:20:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:01 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:05 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:21:09 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442454.5285122, bc7bf9fb-977a-4015-8739-6fa66c1ec210 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:21:09 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] VM Stopped (Lifecycle Event)
Apr 14 03:21:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-9bcca488-12cb-48ee-a68b-4e8fe7b9eaef None None] [instance: bc7bf9fb-977a-4015-8739-6fa66c1ec210] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:21:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:21:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:21:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.148s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357/disk --force-share --output=json" returned: 0 in 0.155s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:21:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=7996MB free_disk=26.167442321777344GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 24d1359d-2dcd-46d7-90f3-3801df9be357 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8cabfa40-3167-4f13-b6f3-4064d326f837 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9b91538e-2a4b-453b-bfb3-5e569da0a3cd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1536MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:21:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.403s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:21:14 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [{"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-24d1359d-2dcd-46d7-90f3-3801df9be357" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:21:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:44 user nova-compute[71323]: INFO nova.compute.manager [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Terminating instance
Apr 14 03:21:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-unplugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] No waiting events found dispatching network-vif-unplugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:21:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-f0e9008a-2fc6-4e0f-b38a-80f455934cf4 req-5f72be52-8290-49c7-bda8-ffc853438f8b service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-unplugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Instance destroyed successfully.
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lazy-loading 'resources' on Instance uuid 24d1359d-2dcd-46d7-90f3-3801df9be357 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:12:21Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestJSON-server-207746006',display_name='tempest-ServerActionsTestJSON-server-207746006',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serveractionstestjson-server-207746006',id=30,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ6y0mvseNBRLhWw4IxRvQ6sM63suvzssGomB2WAhQwXiccO+BpgpoREXX5PCnh6WctOjC8rjVUnY+vy8OsVzTbTpnQ2SqX2vivYxEAAgBe255B5s5TdsyT+Q8Nmdyr13w==',key_name='tempest-keypair-368632906',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:12:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='4075885e26cb4adaad24d3ae6217fe35',ramdisk_id='',reservation_id='r-4xkvk9c0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestJSON-1848627478',owner_user_name='tempest-ServerActionsTestJSON-1848627478-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:12:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7dea23743ba945c4a9d539b8e277c653',uuid=24d1359d-2dcd-46d7-90f3-3801df9be357,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converting VIF {"id": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "address": "fa:16:3e:23:9b:c6", "network": {"id": "e6bbbad8-4a73-4b93-8464-08be8f6681de", "bridge": "br-int", "label": "tempest-ServerActionsTestJSON-283671701-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "4075885e26cb4adaad24d3ae6217fe35", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6ea4c00e-5d", "ovs_interfaceid": "6ea4c00e-5d4b-4375-9b8d-60177a6ec526", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG os_vif [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6ea4c00e-5d, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:21:45 user nova-compute[71323]: INFO os_vif [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:23:9b:c6,bridge_name='br-int',has_traffic_filtering=True,id=6ea4c00e-5d4b-4375-9b8d-60177a6ec526,network=Network(e6bbbad8-4a73-4b93-8464-08be8f6681de),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6ea4c00e-5d')
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Deleting instance files /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357_del
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Deletion of /opt/stack/data/nova/instances/24d1359d-2dcd-46d7-90f3-3801df9be357_del complete
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.compute.manager [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Took 0.84 seconds to destroy the instance on the hypervisor.
Apr 14 03:21:45 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Took 0.43 seconds to deallocate network for instance.
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-fe119513-6cf7-4946-a602-fbcc4f2010e2 req-5f97359e-b11e-4b4c-a82d-9514978aa341 service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-deleted-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:21:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.247s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:45 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Deleted allocations for instance 24d1359d-2dcd-46d7-90f3-3801df9be357
Apr 14 03:21:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7af3c17a-b0a8-4210-a3b6-9fc0bb0188e0 tempest-ServerActionsTestJSON-1848627478 tempest-ServerActionsTestJSON-1848627478-project-member] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.700s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:21:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] Acquiring lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] Lock "24d1359d-2dcd-46d7-90f3-3801df9be357-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] No waiting events found dispatching network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:21:46 user nova-compute[71323]: WARNING nova.compute.manager [req-512fabcc-6bdb-4153-80ca-e87661c96770 req-fb3c3755-14fb-44af-a621-125a977167aa service nova] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Received unexpected event network-vif-plugged-6ea4c00e-5d4b-4375-9b8d-60177a6ec526 for instance with vm_state deleted and task_state None.
Apr 14 03:21:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing instance network info cache due to event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updated VIF entry in instance network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.61", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ab4dc81e-870b-4700-af75-e6fdceb98db4 req-65c9a034-eb31-4eaf-bfc4-a9be59fc52a8 service nova] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing instance network info cache due to event network-changed-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] Acquiring lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] Acquired lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:21:50 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Refreshing network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:21:51 user nova-compute[71323]: INFO nova.compute.manager [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Detaching volume 96f437f9-4897-417a-95e3-097d2ff29476
Apr 14 03:21:51 user nova-compute[71323]: INFO nova.virt.block_device [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Attempting to driver detach volume 96f437f9-4897-417a-95e3-097d2ff29476 from mountpoint /dev/vdb
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Attempting to detach device vdb from instance 8cabfa40-3167-4f13-b6f3-4064d326f837 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:21:51 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:21:51 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:21:51 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:21:51 user nova-compute[71323]:   <serial>96f437f9-4897-417a-95e3-097d2ff29476</serial>
Apr 14 03:21:51 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:21:51 user nova-compute[71323]: </disk>
Apr 14 03:21:51 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:21:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully detached device vdb from instance 8cabfa40-3167-4f13-b6f3-4064d326f837 from the persistent domain config.
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 8cabfa40-3167-4f13-b6f3-4064d326f837 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:21:51 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:21:51 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:21:51 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:21:51 user nova-compute[71323]:   <serial>96f437f9-4897-417a-95e3-097d2ff29476</serial>
Apr 14 03:21:51 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:21:51 user nova-compute[71323]: </disk>
Apr 14 03:21:51 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updated VIF entry in instance network info cache for port a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [{"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cbb060be-b7dd-4c59-8e64-a61c3cb5640c req-601e9344-507c-4878-b625-0a2f3b9e83cd service nova] Releasing lock "refresh_cache-8cabfa40-3167-4f13-b6f3-4064d326f837" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442511.7203035, 8cabfa40-3167-4f13-b6f3-4064d326f837 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:21:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully detached device vdb from instance 8cabfa40-3167-4f13-b6f3-4064d326f837 from the live domain config.
Apr 14 03:21:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JTPOdpZX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:21:51 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[724860bb-26d8-4668-bd42-0a67ff6b69d9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:21:51 user nova-compute[71323]: No targets found.
Apr 14 03:21:51 user nova-compute[71323]: iSNS:
Apr 14 03:21:51 user nova-compute[71323]: No targets found.
Apr 14 03:21:51 user nova-compute[71323]: STATIC:
Apr 14 03:21:51 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:21:51 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:21:51 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:21:51 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476
Apr 14 03:21:51 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:21:51 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:21:51 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:21:51 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:21:51 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:21:51 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36
Apr 14 03:21:51 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:21:51 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:21:51 user nova-compute[71323]: FIRMWARE:
Apr 14 03:21:51 user nova-compute[71323]: No targets found.
Apr 14 03:21:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:21:51 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:21:51 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3c086d5-be9f-422c-9944-2a8584c7d03c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cbc8e56-0cef-44b9-be1c-f1a28f727b1d]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:21:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 (non-flash)
Apr 14 03:21:51 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:21:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae0193c70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d01713c2-ddb9-4bd3-9159-92c2e7239a77]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d8ba261-b845-4e9d-a6b3-b433dae1eeaf]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0084d2a-7af5-4903-8bb7-75b2f7a35c61]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6af4c89-dc13-49f6-a3b9-ae77b71b4f64]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2c41995-578f-4d4f-b16b-697b66cc4a3a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b18ede4f-a3b7-44d1-a109-f91cf7e5c8e4]: (4, ('Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]\nLogout of [sid: 47, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260]
Apr 14 03:21:51 user nova-compute[71323]: Logout of [sid: 47, target: iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476, portal: 172.16.0.220,3260] successful.
Apr 14 03:21:51 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:96f437f9-4897-417a-95e3-097d2ff29476 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1987e4e5-5ddd-4a22-9857-c771091c9bca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.293s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== disconnect_volume: return (294ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG nova.objects.instance [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:21:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-6367bc65-9309-419f-a4a6-f21d79243a8c tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.682s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:21:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:21:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:00 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442505.0593216, 24d1359d-2dcd-46d7-90f3-3801df9be357 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:22:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] VM Stopped (Lifecycle Event)
Apr 14 03:22:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ff7d7bd8-dc86-424b-852b-30c0b247a1df None None] [instance: 24d1359d-2dcd-46d7-90f3-3801df9be357] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:22:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:02 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:03 user nova-compute[71323]: INFO nova.compute.manager [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Terminating instance
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-696c0363-c343-474b-b0a8-6b56b8d3d7d0 req-045b2004-a899-499e-82d4-13be9d7a559e service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-unplugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Instance destroyed successfully.
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.objects.instance [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid ca85d0c8-d192-4bdf-b607-6fe637ad6f81 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:16:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-604221994',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-604221994',id=36,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:18:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-kjvjew37',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:20:17Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=ca85d0c8-d192-4bdf-b607-6fe637ad6f81,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "eac1de6c-0025-4c84-a354-e1418e69740f", "address": "fa:16:3e:02:a9:64", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapeac1de6c-00", "ovs_interfaceid": "eac1de6c-0025-4c84-a354-e1418e69740f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_vif [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapeac1de6c-00, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:03 user nova-compute[71323]: INFO os_vif [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:a9:64,bridge_name='br-int',has_traffic_filtering=True,id=eac1de6c-0025-4c84-a354-e1418e69740f,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapeac1de6c-00')
Apr 14 03:22:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cwEYTBpt', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:22:03 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a02eadd-b3f4-459d-a2e9-2b1750605771]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:22:03 user nova-compute[71323]: No targets found.
Apr 14 03:22:03 user nova-compute[71323]: iSNS:
Apr 14 03:22:03 user nova-compute[71323]: No targets found.
Apr 14 03:22:03 user nova-compute[71323]: STATIC:
Apr 14 03:22:03 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b
Apr 14 03:22:03 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:22:03 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:22:03 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:22:03 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:22:03 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:22:03 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36
Apr 14 03:22:03 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:22:03 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:22:03 user nova-compute[71323]: FIRMWARE:
Apr 14 03:22:03 user nova-compute[71323]: No targets found.
Apr 14 03:22:03 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:22:03 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:22:03 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:22:03 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcf3739a-d672-420d-af90-ea8e46cee004]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c9d9fa6-7807-4288-9b27-310094ee8596]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:22:04 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:22:04 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:22:04 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:22:04 user nova-compute[71323]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b (non-flash)
Apr 14 03:22:04 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:22:04 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccd4a60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1361f1a8-a104-410b-bf7b-9a04f17bdc9b]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d95991f-a3ae-47d8-b338-ce8cc251c815]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa3ea85e-18cc-4dad-9c07-54535f177886]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.049s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5733de68-409b-48e4-a72e-5c485937effb]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIObf54bc07-1b223f65a0bbc8aa', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_bf54bc07', '/dev/disk/by-id/scsi-23162323233663635') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb788b8b-e2f3-43d8-a20f-c213ad6307b4]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecb093ff-d3fc-4ea9-abf0-aa88f9d0ff39]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f9fea4a-4528-4135-ac97-99c782c54a52]: (4, ('Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260]\nLogout of [sid: 41, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260]
Apr 14 03:22:04 user nova-compute[71323]: Logout of [sid: 41, target: iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b, portal: 172.16.0.220,3260] successful.
Apr 14 03:22:04 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:46511b99-537b-43ba-af8f-65b70b89170b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[288133d8-27fc-4ce2-9efb-0dabd046834c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.326s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== disconnect_volume: return (327ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:22:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Deleting instance files /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81_del
Apr 14 03:22:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Deletion of /opt/stack/data/nova/instances/ca85d0c8-d192-4bdf-b607-6fe637ad6f81_del complete
Apr 14 03:22:04 user nova-compute[71323]: INFO nova.compute.manager [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Took 1.22 seconds to destroy the instance on the hypervisor.
Apr 14 03:22:04 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:04 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:04 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Took 0.46 seconds to deallocate network for instance.
Apr 14 03:22:04 user nova-compute[71323]: DEBUG nova.compute.manager [req-c403df20-b6b4-4ba7-9052-11c6c5d3c0cd req-d42e88c0-9950-405c-95b0-cbe86afeba96 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-deleted-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] Acquiring lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] No waiting events found dispatching network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:05 user nova-compute[71323]: WARNING nova.compute.manager [req-092ff57a-684a-4ad6-ad28-ebcceefd21c0 req-1f502419-507f-4ac6-9e7d-c9a498b7ba13 service nova] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Received unexpected event network-vif-plugged-eac1de6c-0025-4c84-a354-e1418e69740f for instance with vm_state active and task_state deleting.
Apr 14 03:22:05 user nova-compute[71323]: INFO nova.compute.manager [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Took 0.93 seconds to detach 1 volumes for instance.
Apr 14 03:22:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:06 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:22:06 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:22:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.280s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:06 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Deleted allocations for instance ca85d0c8-d192-4bdf-b607-6fe637ad6f81
Apr 14 03:22:06 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-05ff6a12-c479-4fce-8eb3-4426456f4eef tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "ca85d0c8-d192-4bdf-b607-6fe637ad6f81" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.080s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:22:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:09 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:22:12 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updating instance_info_cache with network_info: [{"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-bd5b587d-58a9-411b-a755-67adaa6bd6c9" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.265s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:22:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:22:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8289MB free_disk=26.18514633178711GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance bd5b587d-58a9-411b-a755-67adaa6bd6c9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 8cabfa40-3167-4f13-b6f3-4064d326f837 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 9b91538e-2a4b-453b-bfb3-5e569da0a3cd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1280MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:22:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.310s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442523.9156451, ca85d0c8-d192-4bdf-b607-6fe637ad6f81 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:22:18 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] VM Stopped (Lifecycle Event)
Apr 14 03:22:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-b70037bf-d589-4bef-ab44-0de276ab543d None None] [instance: ca85d0c8-d192-4bdf-b607-6fe637ad6f81] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:22:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-changed-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG nova.compute.manager [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Refreshing instance network info cache due to event network-changed-66cc3bdb-6829-4939-95c9-3549b7c39c65. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] Acquiring lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] Acquired lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG nova.network.neutron [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Refreshing network info cache for port 66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:21 user nova-compute[71323]: INFO nova.compute.manager [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Terminating instance
Apr 14 03:22:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updated VIF entry in instance network info cache for port 66cc3bdb-6829-4939-95c9-3549b7c39c65. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updating instance_info_cache with network_info: [{"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.233", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-434ca3c8-6f55-493e-a675-c84be6ff286e req-896ae658-5c78-4c58-a716-9fb7b3bc4d0a service nova] Releasing lock "refresh_cache-9b91538e-2a4b-453b-bfb3-5e569da0a3cd" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-unplugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] No waiting events found dispatching network-vif-unplugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-aef221a6-8199-4fc6-ab86-6d5231c8d2c9 req-ab7d96d6-78ab-4187-bcc9-37563b0d25da service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-unplugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Instance destroyed successfully.
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.objects.instance [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'resources' on Instance uuid 9b91538e-2a4b-453b-bfb3-5e569da0a3cd {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:20:29Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-532403844',display_name='tempest-AttachVolumeNegativeTest-server-532403844',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-532403844',id=39,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHSdJeK0I2g3r6BVK5roXFoHNwGS9/lEYRIk7CP+MH/Kk1cBXHAGb76eRSMrsWN/Ae392cfnXS4PUZ3wHpnCdSrlNAu5O+oD1xGzhmbUo4CzIPnk94ndEYbTDEjXXGjkuQ==',key_name='tempest-keypair-2056266938',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:20:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-009yzo9p',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:20:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=9b91538e-2a4b-453b-bfb3-5e569da0a3cd,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "address": "fa:16:3e:98:ec:25", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66cc3bdb-68", "ovs_interfaceid": "66cc3bdb-6829-4939-95c9-3549b7c39c65", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG os_vif [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap66cc3bdb-68, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:22 user nova-compute[71323]: INFO os_vif [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:98:ec:25,bridge_name='br-int',has_traffic_filtering=True,id=66cc3bdb-6829-4939-95c9-3549b7c39c65,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66cc3bdb-68')
Apr 14 03:22:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Deleting instance files /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd_del
Apr 14 03:22:22 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Deletion of /opt/stack/data/nova/instances/9b91538e-2a4b-453b-bfb3-5e569da0a3cd_del complete
Apr 14 03:22:22 user nova-compute[71323]: INFO nova.compute.manager [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 14 03:22:22 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:22:22 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:23 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Took 0.85 seconds to deallocate network for instance.
Apr 14 03:22:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG nova.compute.manager [req-d25ea8d7-1fe3-40c3-ae92-f0b4f063fa48 req-f4ca9b78-98a7-4857-92b7-3f600086d758 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-deleted-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:22:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.217s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:23 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Deleted allocations for instance 9b91538e-2a4b-453b-bfb3-5e569da0a3cd
Apr 14 03:22:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-db5dbb8a-cfa6-4cbc-8539-90a5cca7e716 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.920s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] Acquiring lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] Lock "9b91538e-2a4b-453b-bfb3-5e569da0a3cd-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:24 user nova-compute[71323]: DEBUG nova.compute.manager [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] No waiting events found dispatching network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:24 user nova-compute[71323]: WARNING nova.compute.manager [req-a70c410f-8bec-4fa4-9793-7979d99c4ae8 req-87b8eeab-46d9-4fd9-b1df-25fc0f2335d4 service nova] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Received unexpected event network-vif-plugged-66cc3bdb-6829-4939-95c9-3549b7c39c65 for instance with vm_state deleted and task_state None.
Apr 14 03:22:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:37 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442542.3468077, 9b91538e-2a4b-453b-bfb3-5e569da0a3cd => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:22:37 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] VM Stopped (Lifecycle Event)
Apr 14 03:22:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-461ed45f-e7b6-4750-a8e3-3725f8e3b148 None None] [instance: 9b91538e-2a4b-453b-bfb3-5e569da0a3cd] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:22:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:45 user nova-compute[71323]: INFO nova.compute.manager [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Terminating instance
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-808b05c0-370f-4bf2-bf0b-265054bdf3f8 req-57b90542-8864-415c-8cbe-f2af005f2c8b service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-unplugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Instance destroyed successfully.
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'resources' on Instance uuid 8cabfa40-3167-4f13-b6f3-4064d326f837 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:17:47Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-252373748',display_name='tempest-TestMinimumBasicScenario-server-252373748',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-252373748',id=37,image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFLpvRH/iiMHYTrjdAzOT7swMWulL8zixU7ppruKTH3LsxMEI1RMZv3PNx+fzQlWx2Y295WnVKwNn+mPfk5MJDyTHw+kzZIW87C2e8Z/iARwbfZzvewxzYpvwTsj4RrU0g==',key_name='tempest-TestMinimumBasicScenario-38758631',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:17:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-dpqwyg0x',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='3af87ed2-d2c6-4608-b72f-8b0f91e4ca86',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:20:06Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=8cabfa40-3167-4f13-b6f3-4064d326f837,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "address": "fa:16:3e:71:a9:a9", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa478a9ea-0f", "ovs_interfaceid": "a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG os_vif [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa478a9ea-0f, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:45 user nova-compute[71323]: INFO os_vif [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:71:a9:a9,bridge_name='br-int',has_traffic_filtering=True,id=a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa478a9ea-0f')
Apr 14 03:22:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Deleting instance files /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837_del
Apr 14 03:22:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Deletion of /opt/stack/data/nova/instances/8cabfa40-3167-4f13-b6f3-4064d326f837_del complete
Apr 14 03:22:45 user nova-compute[71323]: INFO nova.compute.manager [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Took 0.85 seconds to destroy the instance on the hypervisor.
Apr 14 03:22:45 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:22:45 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:46 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Took 0.55 seconds to deallocate network for instance.
Apr 14 03:22:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-788b4ffa-1863-4858-b11c-1a8c5a561531 req-d0487823-24f8-44e5-8189-3324ea89c2ea service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-deleted-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:22:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.188s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:46 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Deleted allocations for instance 8cabfa40-3167-4f13-b6f3-4064d326f837
Apr 14 03:22:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2391a35-130f-410d-ac8e-cf4410dd24ff tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.780s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] Acquiring lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] Lock "8cabfa40-3167-4f13-b6f3-4064d326f837-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] No waiting events found dispatching network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:47 user nova-compute[71323]: WARNING nova.compute.manager [req-eeae9fbd-e70c-4f02-aa55-b1f7a87473e9 req-d34c24ef-070d-4f25-8e7c-76ba6b69ea13 service nova] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Received unexpected event network-vif-plugged-a478a9ea-0fba-4d3e-8fa1-9ba6371c01a0 for instance with vm_state deleted and task_state None.
Apr 14 03:22:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:53 user nova-compute[71323]: INFO nova.compute.manager [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Terminating instance
Apr 14 03:22:53 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:22:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-unplugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-unplugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-7b9598d4-e309-4ce2-b148-f38e0c1a20d3 req-d5229421-8bb1-4153-a4f2-0d30640e571c service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-unplugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Instance destroyed successfully.
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid bd5b587d-58a9-411b-a755-67adaa6bd6c9 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:13:31Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-289155264',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-289155264',id=33,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:13:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-9yzst8e4',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:15:28Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=bd5b587d-58a9-411b-a755-67adaa6bd6c9,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "address": "fa:16:3e:48:d8:5c", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52fd899f-49", "ovs_interfaceid": "52fd899f-496c-41d3-a9e0-6574a27e3b1b", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG os_vif [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52fd899f-49, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:22:54 user nova-compute[71323]: INFO os_vif [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:48:d8:5c,bridge_name='br-int',has_traffic_filtering=True,id=52fd899f-496c-41d3-a9e0-6574a27e3b1b,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52fd899f-49')
Apr 14 03:22:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Deleting instance files /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9_del
Apr 14 03:22:54 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Deletion of /opt/stack/data/nova/instances/bd5b587d-58a9-411b-a755-67adaa6bd6c9_del complete
Apr 14 03:22:54 user nova-compute[71323]: INFO nova.compute.manager [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Took 0.88 seconds to destroy the instance on the hypervisor.
Apr 14 03:22:54 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:22:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:22:55 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Took 0.99 seconds to deallocate network for instance.
Apr 14 03:22:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:22:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.178s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:55 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Deleted allocations for instance bd5b587d-58a9-411b-a755-67adaa6bd6c9
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7c25ce80-fa68-469d-a87b-4b86a13d2cfc tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.219s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:56 user nova-compute[71323]: WARNING nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state deleted and task_state None.
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:56 user nova-compute[71323]: WARNING nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state deleted and task_state None.
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:56 user nova-compute[71323]: WARNING nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state deleted and task_state None.
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Acquiring lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] Lock "bd5b587d-58a9-411b-a755-67adaa6bd6c9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] No waiting events found dispatching network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:22:56 user nova-compute[71323]: WARNING nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received unexpected event network-vif-plugged-52fd899f-496c-41d3-a9e0-6574a27e3b1b for instance with vm_state deleted and task_state None.
Apr 14 03:22:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-45fc63b8-a2c8-4e2a-918a-6ae317349ed4 req-16c359d6-deea-4739-b0f0-6ee409638d4b service nova] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Received event network-vif-deleted-52fd899f-496c-41d3-a9e0-6574a27e3b1b {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:22:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:00 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442565.7883239, 8cabfa40-3167-4f13-b6f3-4064d326f837 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] VM Stopped (Lifecycle Event)
Apr 14 03:23:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d5e01bc6-5f63-490f-8b40-566e9275b09b None None] [instance: 8cabfa40-3167-4f13-b6f3-4064d326f837] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:23:08 user nova-compute[71323]: INFO nova.compute.claims [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Claim successful on node user
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.285s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:23:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:23:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Creating image(s)
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "/opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "/opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "/opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.008s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:08 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk 1073741824" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Checking if we can resize image /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442574.577094, bd5b587d-58a9-411b-a755-67adaa6bd6c9 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:09 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] VM Stopped (Lifecycle Event)
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7cfae05a-ff5f-4ed8-8541-e701fba27a93 None None] [instance: bd5b587d-58a9-411b-a755-67adaa6bd6c9] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Cannot resize image /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.objects.instance [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lazy-loading 'migration_context' on Instance uuid 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Ensure instance console log exists: /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:09 user nova-compute[71323]: DEBUG nova.policy [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '85a2f927a9b94b6a8e9d91932bab0af6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1ccb5ceb9f7641a9b6265de1f3c38c89', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:23:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Successfully created port: 587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Successfully updated port: 587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquired lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-changed-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.compute.manager [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Refreshing instance network info cache due to event network-changed-587e1221-a4b5-4e33-8621-2606be640411. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] Acquiring lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.neutron [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Updating instance_info_cache with network_info: [{"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Releasing lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Instance network_info: |[{"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] Acquired lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Refreshing network info cache for port 587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Start _get_guest_xml network_info=[{"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:23:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:11 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1553175621',display_name='tempest-VolumesNegativeTest-instance-1553175621',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1553175621',id=40,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1ccb5ceb9f7641a9b6265de1f3c38c89',ramdisk_id='',reservation_id='r-x4t4rt88',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1844013062',owner_user_name='tempest-VolumesNegativeTest-1844013062-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:23:09Z,user_data=None,user_id='85a2f927a9b94b6a8e9d91932bab0af6',uuid=864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converting VIF {"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.objects.instance [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lazy-loading 'pci_devices' on Instance uuid 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:23:11 user nova-compute[71323]:   <uuid>864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420</uuid>
Apr 14 03:23:11 user nova-compute[71323]:   <name>instance-00000028</name>
Apr 14 03:23:11 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:23:11 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:23:11 user nova-compute[71323]:   <metadata>
Apr 14 03:23:11 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:23:11 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:name>tempest-VolumesNegativeTest-instance-1553175621</nova:name>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:23:11</nova:creationTime>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:23:11 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:23:11 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:owner>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:user uuid="85a2f927a9b94b6a8e9d91932bab0af6">tempest-VolumesNegativeTest-1844013062-project-member</nova:user>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:project uuid="1ccb5ceb9f7641a9b6265de1f3c38c89">tempest-VolumesNegativeTest-1844013062</nova:project>
Apr 14 03:23:11 user nova-compute[71323]:       </nova:owner>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:23:11 user nova-compute[71323]:       <nova:ports>
Apr 14 03:23:11 user nova-compute[71323]:         <nova:port uuid="587e1221-a4b5-4e33-8621-2606be640411">
Apr 14 03:23:11 user nova-compute[71323]:           <nova:ip type="fixed" address="192.168.233.235" ipVersion="4"/>
Apr 14 03:23:11 user nova-compute[71323]:         </nova:port>
Apr 14 03:23:11 user nova-compute[71323]:       </nova:ports>
Apr 14 03:23:11 user nova-compute[71323]:     </nova:instance>
Apr 14 03:23:11 user nova-compute[71323]:   </metadata>
Apr 14 03:23:11 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:23:11 user nova-compute[71323]:     <system>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="serial">864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420</entry>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="uuid">864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420</entry>
Apr 14 03:23:11 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:23:11 user nova-compute[71323]:     </system>
Apr 14 03:23:11 user nova-compute[71323]:   </sysinfo>
Apr 14 03:23:11 user nova-compute[71323]:   <os>
Apr 14 03:23:11 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:23:11 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:23:11 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:23:11 user nova-compute[71323]:   </os>
Apr 14 03:23:11 user nova-compute[71323]:   <features>
Apr 14 03:23:11 user nova-compute[71323]:     <acpi/>
Apr 14 03:23:11 user nova-compute[71323]:     <apic/>
Apr 14 03:23:11 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:23:11 user nova-compute[71323]:   </features>
Apr 14 03:23:11 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:23:11 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:23:11 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:23:11 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:23:11 user nova-compute[71323]:   </clock>
Apr 14 03:23:11 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:23:11 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:23:11 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:23:11 user nova-compute[71323]:   </cpu>
Apr 14 03:23:11 user nova-compute[71323]:   <devices>
Apr 14 03:23:11 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:23:11 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:23:11 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk"/>
Apr 14 03:23:11 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:23:11 user nova-compute[71323]:     </disk>
Apr 14 03:23:11 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:23:11 user nova-compute[71323]:       <mac address="fa:16:3e:99:34:d1"/>
Apr 14 03:23:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:23:11 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:23:11 user nova-compute[71323]:       <target dev="tap587e1221-a4"/>
Apr 14 03:23:11 user nova-compute[71323]:     </interface>
Apr 14 03:23:11 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:23:11 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/console.log" append="off"/>
Apr 14 03:23:11 user nova-compute[71323]:     </serial>
Apr 14 03:23:11 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:23:11 user nova-compute[71323]:     <video>
Apr 14 03:23:11 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:23:11 user nova-compute[71323]:     </video>
Apr 14 03:23:11 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:23:11 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:23:11 user nova-compute[71323]:     </rng>
Apr 14 03:23:11 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:23:11 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:23:11 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:23:11 user nova-compute[71323]:     </memballoon>
Apr 14 03:23:11 user nova-compute[71323]:   </devices>
Apr 14 03:23:11 user nova-compute[71323]: </domain>
Apr 14 03:23:11 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1553175621',display_name='tempest-VolumesNegativeTest-instance-1553175621',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1553175621',id=40,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1ccb5ceb9f7641a9b6265de1f3c38c89',ramdisk_id='',reservation_id='r-x4t4rt88',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-1844013062',owner_user_name='tempest-VolumesNegativeTest-1844013062-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:23:09Z,user_data=None,user_id='85a2f927a9b94b6a8e9d91932bab0af6',uuid=864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converting VIF {"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG os_vif [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap587e1221-a4, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap587e1221-a4, col_values=(('external_ids', {'iface-id': '587e1221-a4b5-4e33-8621-2606be640411', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:99:34:d1', 'vm-uuid': '864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:11 user nova-compute[71323]: INFO os_vif [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4')
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] No VIF found with MAC fa:16:3e:99:34:d1, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:23:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Updated VIF entry in instance network info cache for port 587e1221-a4b5-4e33-8621-2606be640411. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Updating instance_info_cache with network_info: [{"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-47205489-2641-4948-86b7-b3e70acc4e12 req-b7233728-87ec-49a3-bbb9-dd3fd1ebcc11 service nova] Releasing lock "refresh_cache-864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:12 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:13 user nova-compute[71323]: INFO nova.compute.manager [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Detaching volume c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: INFO nova.virt.block_device [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Attempting to driver detach volume c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 from mountpoint /dev/vdb
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Attempting to detach device vdb from instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:23:13 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:23:13 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:23:13 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:23:13 user nova-compute[71323]:   <serial>c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36</serial>
Apr 14 03:23:13 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:23:13 user nova-compute[71323]: </disk>
Apr 14 03:23:13 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:23:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 from the persistent domain config.
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:23:13 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:23:13 user nova-compute[71323]:   <source dev="/dev/sde"/>
Apr 14 03:23:13 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:23:13 user nova-compute[71323]:   <serial>c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36</serial>
Apr 14 03:23:13 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:23:13 user nova-compute[71323]: </disk>
Apr 14 03:23:13 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] No waiting events found dispatching network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:13 user nova-compute[71323]: WARNING nova.compute.manager [req-a8cbce75-4907-4f67-8e47-9287a3350e01 req-4519a634-0977-4808-b128-8826a6e97d6c service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received unexpected event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 for instance with vm_state building and task_state spawning.
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442593.7025871, 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:23:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 from the live domain config.
Apr 14 03:23:13 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YIukGbUA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:23:13 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7f44602-a2d8-4e04-b2cf-0efdbaabfff8]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:23:13 user nova-compute[71323]: No targets found.
Apr 14 03:23:13 user nova-compute[71323]: iSNS:
Apr 14 03:23:13 user nova-compute[71323]: No targets found.
Apr 14 03:23:13 user nova-compute[71323]: STATIC:
Apr 14 03:23:13 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:23:13 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:23:13 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:23:13 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36
Apr 14 03:23:13 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:23:13 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:23:13 user nova-compute[71323]: FIRMWARE:
Apr 14 03:23:13 user nova-compute[71323]: No targets found.
Apr 14 03:23:13 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:23:13 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:23:13 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[100db2ba-f6d0-4eb8-b231-c3b646791953]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bbea37d-1e1a-490c-b192-9bd9a1a8c7a6]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:23:13 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:23:13 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:23:13 user nova-compute[71323]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 (non-flash)
Apr 14 03:23:13 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b003e0820>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36'): ({'sde'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Removing single pathed devices sde {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e37bb445-1248-49e9-9f70-ebcbc82b11e5]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa66f15e-b5a4-416c-991d-c3847e986de0]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flushing IO for device /dev/sde {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d9db71e-a363-4e56-b14c-efca3c06a290]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.039s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[796adfeb-45df-48d8-9c46-a77c93b8ca47]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] SCSI volumes sde have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[72ae6af8-0a5e-4eb1-89d4-87accb2d73b4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[829041d1-e0e5-4003-87de-496f3e11a01a]: (4, ('Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260]\nLogout of [sid: 48, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260]
Apr 14 03:23:14 user nova-compute[71323]: Logout of [sid: 48, target: iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36, portal: 172.16.0.220,3260] successful.
Apr 14 03:23:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c2c4c1de-825b-42bd-a2a1-c7ed4e4eba36 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c59fddd3-0fa3-486a-ae2f-adf939fdfdca]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.342s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== disconnect_volume: return (343ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.153s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442595.434834, 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] VM Resumed (Lifecycle Event)
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] No waiting events found dispatching network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:15 user nova-compute[71323]: WARNING nova.compute.manager [req-c7585c57-a5d5-437e-8f89-073732140965 req-d3fdc29d-3bc6-4317-8289-79bd4067ec97 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received unexpected event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 for instance with vm_state building and task_state spawning.
Apr 14 03:23:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8469MB free_disk=26.304840087890625GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Instance spawned successfully.
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442595.4359915, 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] VM Started (Lifecycle Event)
Apr 14 03:23:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5a25cea7-36c8-46ee-81c5-2518d6d2cb35 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 2.719s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1024MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:23:15 user nova-compute[71323]: INFO nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Took 6.94 seconds to spawn the instance on the hypervisor.
Apr 14 03:23:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:16 user nova-compute[71323]: INFO nova.compute.manager [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Took 7.55 seconds to build instance.
Apr 14 03:23:16 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:23:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-31c8b5be-ae0f-4254-b233-9000830b21c7 tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.642s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:16 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:23:16 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:23:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.327s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating instance_info_cache with network_info: [{"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:23:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:23:38 user nova-compute[71323]: INFO nova.compute.claims [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Claim successful on node user
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.261s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:23:38 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:23:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:23:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Creating image(s)
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "/opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "/opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "/opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.policy [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '12199a43298942898a2e694ee2c5230e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0bd451b3f6734fe3ac0a8e8a853bb9eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.part --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.virt.images [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] eed6e9a9-28a5-4022-b718-e7f64bbf7dfd was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.part /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Successfully created port: 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.part /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.converted" returned: 0 in 0.289s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:39 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e.converted --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.983s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e,backing_fmt=raw /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Successfully updated port: 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e,backing_fmt=raw /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk 1073741824" returned: 0 in 0.050s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "0286d9d01d1fe187421499cd39dcada15ca6c93e" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.192s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing instance network info cache due to event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Cannot resize image /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'migration_context' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Ensure instance console log exists: /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e93e76a0-b5d7-491d-84d9-0663631c03f9 req-adc79328-d867-4a73-b9a5-5eafceee1e72 service nova] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:23:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance network_info: |[{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Start _get_guest_xml network_info=[{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T03:23:36Z,direct_url=<?>,disk_format='qcow2',id=eed6e9a9-28a5-4022-b718-e7f64bbf7dfd,min_disk=0,min_ram=0,name='tempest-scenario-img--2042960710',owner='0bd451b3f6734fe3ac0a8e8a853bb9eb',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T03:23:37Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': 'eed6e9a9-28a5-4022-b718-e7f64bbf7dfd'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:23:41 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:41 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T03:23:36Z,direct_url=<?>,disk_format='qcow2',id=eed6e9a9-28a5-4022-b718-e7f64bbf7dfd,min_disk=0,min_ram=0,name='tempest-scenario-img--2042960710',owner='0bd451b3f6734fe3ac0a8e8a853bb9eb',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T03:23:37Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1949831724',display_name='tempest-TestMinimumBasicScenario-server-1949831724',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1949831724',id=41,image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbABr/vH+9uc5wQJ3ZeT8Ydil5OovYfdLqGJBQSVbmCHz/2Slj13rAwCGE1mbPPo0/CwXQms3UgRTIS6Sa1TpYWM58n3w3eYIRd+WO/4CHQY+ShdxeZ/KOMF7laMdnfUw==',key_name='tempest-TestMinimumBasicScenario-35983132',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-3ckb0aae',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:23:39Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=f8042c4c-0ff6-4533-8f59-de5c41ede33f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'pci_devices' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:23:41 user nova-compute[71323]:   <uuid>f8042c4c-0ff6-4533-8f59-de5c41ede33f</uuid>
Apr 14 03:23:41 user nova-compute[71323]:   <name>instance-00000029</name>
Apr 14 03:23:41 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:23:41 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:23:41 user nova-compute[71323]:   <metadata>
Apr 14 03:23:41 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:23:41 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:name>tempest-TestMinimumBasicScenario-server-1949831724</nova:name>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:23:41</nova:creationTime>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:23:41 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:23:41 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:owner>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:user uuid="12199a43298942898a2e694ee2c5230e">tempest-TestMinimumBasicScenario-144187586-project-member</nova:user>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:project uuid="0bd451b3f6734fe3ac0a8e8a853bb9eb">tempest-TestMinimumBasicScenario-144187586</nova:project>
Apr 14 03:23:41 user nova-compute[71323]:       </nova:owner>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:root type="image" uuid="eed6e9a9-28a5-4022-b718-e7f64bbf7dfd"/>
Apr 14 03:23:41 user nova-compute[71323]:       <nova:ports>
Apr 14 03:23:41 user nova-compute[71323]:         <nova:port uuid="8a4c167f-3166-4f90-91c9-1a1d5be3c92d">
Apr 14 03:23:41 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Apr 14 03:23:41 user nova-compute[71323]:         </nova:port>
Apr 14 03:23:41 user nova-compute[71323]:       </nova:ports>
Apr 14 03:23:41 user nova-compute[71323]:     </nova:instance>
Apr 14 03:23:41 user nova-compute[71323]:   </metadata>
Apr 14 03:23:41 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:23:41 user nova-compute[71323]:     <system>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="serial">f8042c4c-0ff6-4533-8f59-de5c41ede33f</entry>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="uuid">f8042c4c-0ff6-4533-8f59-de5c41ede33f</entry>
Apr 14 03:23:41 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:23:41 user nova-compute[71323]:     </system>
Apr 14 03:23:41 user nova-compute[71323]:   </sysinfo>
Apr 14 03:23:41 user nova-compute[71323]:   <os>
Apr 14 03:23:41 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:23:41 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:23:41 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:23:41 user nova-compute[71323]:   </os>
Apr 14 03:23:41 user nova-compute[71323]:   <features>
Apr 14 03:23:41 user nova-compute[71323]:     <acpi/>
Apr 14 03:23:41 user nova-compute[71323]:     <apic/>
Apr 14 03:23:41 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:23:41 user nova-compute[71323]:   </features>
Apr 14 03:23:41 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:23:41 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:23:41 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:23:41 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:23:41 user nova-compute[71323]:   </clock>
Apr 14 03:23:41 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:23:41 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:23:41 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:23:41 user nova-compute[71323]:   </cpu>
Apr 14 03:23:41 user nova-compute[71323]:   <devices>
Apr 14 03:23:41 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:23:41 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:23:41 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk"/>
Apr 14 03:23:41 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:23:41 user nova-compute[71323]:     </disk>
Apr 14 03:23:41 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:23:41 user nova-compute[71323]:       <mac address="fa:16:3e:94:e8:44"/>
Apr 14 03:23:41 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:23:41 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:23:41 user nova-compute[71323]:       <target dev="tap8a4c167f-31"/>
Apr 14 03:23:41 user nova-compute[71323]:     </interface>
Apr 14 03:23:41 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:23:41 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/console.log" append="off"/>
Apr 14 03:23:41 user nova-compute[71323]:     </serial>
Apr 14 03:23:41 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:23:41 user nova-compute[71323]:     <video>
Apr 14 03:23:41 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:23:41 user nova-compute[71323]:     </video>
Apr 14 03:23:41 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:23:41 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:23:41 user nova-compute[71323]:     </rng>
Apr 14 03:23:41 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:23:41 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:23:41 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:23:41 user nova-compute[71323]:     </memballoon>
Apr 14 03:23:41 user nova-compute[71323]:   </devices>
Apr 14 03:23:41 user nova-compute[71323]: </domain>
Apr 14 03:23:41 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1949831724',display_name='tempest-TestMinimumBasicScenario-server-1949831724',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1949831724',id=41,image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbABr/vH+9uc5wQJ3ZeT8Ydil5OovYfdLqGJBQSVbmCHz/2Slj13rAwCGE1mbPPo0/CwXQms3UgRTIS6Sa1TpYWM58n3w3eYIRd+WO/4CHQY+ShdxeZ/KOMF7laMdnfUw==',key_name='tempest-TestMinimumBasicScenario-35983132',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-3ckb0aae',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:23:39Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=f8042c4c-0ff6-4533-8f59-de5c41ede33f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG os_vif [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8a4c167f-31, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8a4c167f-31, col_values=(('external_ids', {'iface-id': '8a4c167f-3166-4f90-91c9-1a1d5be3c92d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:94:e8:44', 'vm-uuid': 'f8042c4c-0ff6-4533-8f59-de5c41ede33f'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:41 user nova-compute[71323]: INFO os_vif [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31')
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:23:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No VIF found with MAC fa:16:3e:94:e8:44, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:42 user nova-compute[71323]: WARNING nova.compute.manager [req-b8245910-d9d4-45ca-8aa4-8c0d5234d42c req-6b66b368-ee9b-453c-b6bd-8c2e9f1f6eb6 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state building and task_state spawning.
Apr 14 03:23:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:44 user nova-compute[71323]: INFO nova.compute.manager [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Terminating instance
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-b2eb1d45-8eb7-4996-829f-291bede29292 req-23633bc4-9469-409c-9af5-0babb8a8f7b6 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-unplugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:44 user nova-compute[71323]: WARNING nova.compute.manager [req-575914a9-8356-48d5-881e-a466867113df req-43f8363e-aa53-4ae6-a9b7-1950f6989e9c service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state building and task_state spawning.
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442624.9385235, f8042c4c-0ff6-4533-8f59-de5c41ede33f => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:44 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] VM Resumed (Lifecycle Event)
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:23:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance spawned successfully.
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:23:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442624.9421225, f8042c4c-0ff6-4533-8f59-de5c41ede33f => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] VM Started (Lifecycle Event)
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Instance destroyed successfully.
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.objects.instance [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:07:57Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1770273062',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1770273062',id=22,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:09:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-reuivjtu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:11:45Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=2692a422-82e6-4f3b-b91e-6a2ea28e4d0a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "address": "fa:16:3e:86:2f:f6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3b7ac3bb-62", "ovs_interfaceid": "3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_vif [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Took 6.07 seconds to spawn the instance on the hypervisor.
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3b7ac3bb-62, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:23:45 user nova-compute[71323]: INFO os_vif [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:86:2f:f6,bridge_name='br-int',has_traffic_filtering=True,id=3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3b7ac3bb-62')
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kMFlEcQj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:23:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[daeacc65-141b-44a7-ada4-2e54ccdf1212]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:23:45 user nova-compute[71323]: No targets found.
Apr 14 03:23:45 user nova-compute[71323]: iSNS:
Apr 14 03:23:45 user nova-compute[71323]: No targets found.
Apr 14 03:23:45 user nova-compute[71323]: STATIC:
Apr 14 03:23:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c
Apr 14 03:23:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:23:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:23:45 user nova-compute[71323]: FIRMWARE:
Apr 14 03:23:45 user nova-compute[71323]: No targets found.
Apr 14 03:23:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:23:45 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:23:45 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb56befd-849e-40e8-83c3-d9c990426b69]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Took 6.64 seconds to build instance.
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[63fca9d8-662d-4540-ae67-a77c3f90eae2]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:23:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c (non-flash)
Apr 14 03:23:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ae0192170>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c'): ({'sdf'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Removing single pathed devices sdf {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a6b34dab-f85a-412c-a70b-b816bc10f44e tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.735s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d01f0738-5f9c-4d58-a482-b2571931062c]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3329832-7cd7-4e67-a050-3e10d8168daf]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Flushing IO for device /dev/sdf {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[53bf5be8-de44-4857-8029-8f7d4b202217]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f3a5a35-6033-4281-a76c-0a3481b02593]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] SCSI volumes sdf have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbfbeaa5-3677-419c-972c-997527ff38a6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ed76b3e-1583-4cff-a2e0-9a8d3a1e2afa]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260]
Apr 14 03:23:45 user nova-compute[71323]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c, portal: 172.16.0.220,3260] successful.
Apr 14 03:23:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0af4cdb2-c179-4459-8ef7-43518caaaf7c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a582db0-27ae-457f-9e92-f1be3f11fb0d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] <== disconnect_volume: return (309ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Deleting instance files /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a_del
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Deletion of /opt/stack/data/nova/instances/2692a422-82e6-4f3b-b91e-6a2ea28e4d0a_del complete
Apr 14 03:23:45 user nova-compute[71323]: INFO nova.compute.manager [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Took 1.33 seconds to destroy the instance on the hypervisor.
Apr 14 03:23:45 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:23:45 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:23:46 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:23:46 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Took 0.89 seconds to deallocate network for instance.
Apr 14 03:23:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] Acquiring lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] No waiting events found dispatching network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:23:46 user nova-compute[71323]: WARNING nova.compute.manager [req-6803cb4f-4a31-40b7-8932-27cb13f54ce0 req-0431431e-7ef2-4782-a4c2-a9e7e7ad522e service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received unexpected event network-vif-plugged-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 for instance with vm_state active and task_state deleting.
Apr 14 03:23:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-af5130cb-72f9-4a8b-95c8-149c0556e3e2 req-9427057b-07dd-4df2-b628-c26a2a549f12 service nova] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Received event network-vif-deleted-3b7ac3bb-62fa-4afb-8ede-abd8fa94c0b5 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:23:47 user nova-compute[71323]: INFO nova.compute.manager [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Took 0.83 seconds to detach 1 volumes for instance.
Apr 14 03:23:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:23:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:23:47 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:23:47 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:23:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.214s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:47 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Deleted allocations for instance 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a
Apr 14 03:23:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-7d9db6a0-74eb-4c7e-9dc5-6048c003518a tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "2692a422-82e6-4f3b-b91e-6a2ea28e4d0a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.435s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:23:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:23:55 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:23:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:00 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442625.168347, 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:24:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] VM Stopped (Lifecycle Event)
Apr 14 03:24:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a9e6ed4e-5760-4cf7-a1c5-a3861c9d5da9 None None] [instance: 2692a422-82e6-4f3b-b91e-6a2ea28e4d0a] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:24:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:03 user nova-compute[71323]: INFO nova.compute.manager [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Terminating instance
Apr 14 03:24:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-unplugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] No waiting events found dispatching network-vif-unplugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:24:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-f437f814-3ca6-4e4f-aa02-b0f83fc49e73 req-8d136185-b733-4ca5-96e2-1b5be117b12f service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-unplugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:24:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Instance destroyed successfully.
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'resources' on Instance uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:18:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-260481439',display_name='tempest-AttachVolumeNegativeTest-server-260481439',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-260481439',id=38,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP/ufTFDXnGMjwB2FSg+A9mMX0vCesLJiAzD3NMUcRVyCg0qAk7buF7y/zaESmh9YTRgnJRFZQSsoliYulIJhfUVJEsLn5+ezAqOMz+G6LfzgivXohWdPYf/eGFCH+h13A==',key_name='tempest-keypair-1704381483',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:18:31Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-mhfzow8a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:18:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=04737acf-2aa7-4f8c-a33d-fa2b50ea29d7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "address": "fa:16:3e:ce:8b:f2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.98", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap4ceedbf8-a6", "ovs_interfaceid": "4ceedbf8-a65d-4a77-b13c-837ae6290ea1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG os_vif [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap4ceedbf8-a6, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:04 user nova-compute[71323]: INFO os_vif [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ce:8b:f2,bridge_name='br-int',has_traffic_filtering=True,id=4ceedbf8-a65d-4a77-b13c-837ae6290ea1,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4ceedbf8-a6')
Apr 14 03:24:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Deleting instance files /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7_del
Apr 14 03:24:04 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Deletion of /opt/stack/data/nova/instances/04737acf-2aa7-4f8c-a33d-fa2b50ea29d7_del complete
Apr 14 03:24:04 user nova-compute[71323]: INFO nova.compute.manager [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Took 0.87 seconds to destroy the instance on the hypervisor.
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:04 user nova-compute[71323]: WARNING nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] While synchronizing instance power states, found 4 instances in the database and 3 instances on the hypervisor.
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.028s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.032s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.031s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:24:05 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Took 0.68 seconds to deallocate network for instance.
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.181s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Deleted allocations for instance 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-9468725b-346d-479f-a82d-8e056782870e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.898s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.520s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:05 user nova-compute[71323]: INFO nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] During sync_power_state the instance has a pending task (deleting). Skip.
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] Acquiring lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] Lock "04737acf-2aa7-4f8c-a33d-fa2b50ea29d7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] No waiting events found dispatching network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:24:05 user nova-compute[71323]: WARNING nova.compute.manager [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received unexpected event network-vif-plugged-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 for instance with vm_state deleted and task_state None.
Apr 14 03:24:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-797aaaff-eddf-4d86-b38a-2bbb1dcc5fe4 req-8dac16e0-afc7-40a7-99b8-711070fb2dc4 service nova] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Received event network-vif-deleted-4ceedbf8-a65d-4a77-b13c-837ae6290ea1 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:06 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:24:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:24:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:14 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:24:14 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8429MB free_disk=26.266521453857422GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.348s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:24:16 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [{"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:24:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:24:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:19 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442644.3849945, 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:24:19 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] VM Stopped (Lifecycle Event)
Apr 14 03:24:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3b30ff49-fe33-4c12-bf6e-a2006858c8a2 None None] [instance: 04737acf-2aa7-4f8c-a33d-fa2b50ea29d7] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:24:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:24:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:34 user nova-compute[71323]: INFO nova.compute.manager [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Terminating instance
Apr 14 03:24:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-unplugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] No waiting events found dispatching network-vif-unplugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-3b78b11f-6d1c-4c53-a8d6-def17b35327c req-a1ee9f87-f171-43c0-b3ba-a890c7cbd304 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-unplugged-22f727c9-9207-46e6-a471-48de5bf03677 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Instance destroyed successfully.
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lazy-loading 'resources' on Instance uuid d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:04:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-514488998',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-514488998',id=18,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:05:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='bd2c5b2aafaf4ca7bf62e988ac726658',ramdisk_id='',reservation_id='r-8i0b0x43',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:06:56Z,user_data=None,user_id='d666936eb0ee43a2ad9e17a12e6e273d',uuid=d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converting VIF {"id": "22f727c9-9207-46e6-a471-48de5bf03677", "address": "fa:16:3e:db:67:a6", "network": {"id": "de8764bb-1ca4-4c30-8dee-bce850658327", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-1169617420-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "bd2c5b2aafaf4ca7bf62e988ac726658", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22f727c9-92", "ovs_interfaceid": "22f727c9-9207-46e6-a471-48de5bf03677", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG os_vif [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap22f727c9-92, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:35 user nova-compute[71323]: INFO os_vif [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:db:67:a6,bridge_name='br-int',has_traffic_filtering=True,id=22f727c9-9207-46e6-a471-48de5bf03677,network=Network(de8764bb-1ca4-4c30-8dee-bce850658327),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22f727c9-92')
Apr 14 03:24:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Deleting instance files /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e_del
Apr 14 03:24:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Deletion of /opt/stack/data/nova/instances/d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e_del complete
Apr 14 03:24:35 user nova-compute[71323]: INFO nova.compute.manager [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 14 03:24:35 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:24:35 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:24:36 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Took 0.44 seconds to deallocate network for instance.
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-0b35f89c-198c-46c4-acf6-2f0abd55e5f0 req-8201de75-86d7-4a45-9328-a968f304f491 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-deleted-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:36 user nova-compute[71323]: INFO nova.compute.manager [req-0b35f89c-198c-46c4-acf6-2f0abd55e5f0 req-8201de75-86d7-4a45-9328-a968f304f491 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Neutron deleted interface 22f727c9-9207-46e6-a471-48de5bf03677; detaching it from the instance and deleting it from the info cache
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-0b35f89c-198c-46c4-acf6-2f0abd55e5f0 req-8201de75-86d7-4a45-9328-a968f304f491 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.compute.manager [req-0b35f89c-198c-46c4-acf6-2f0abd55e5f0 req-8201de75-86d7-4a45-9328-a968f304f491 service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Detach interface failed, port_id=22f727c9-9207-46e6-a471-48de5bf03677, reason: Instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:24:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.164s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:36 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Deleted allocations for instance d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e
Apr 14 03:24:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c2432d85-e5ae-4b2a-8758-c29bc3d9aeb7 tempest-ServerBootFromVolumeStableRescueTest-1400694371 tempest-ServerBootFromVolumeStableRescueTest-1400694371-project-member] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.690s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] Acquiring lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] Lock "d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] No waiting events found dispatching network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:24:37 user nova-compute[71323]: WARNING nova.compute.manager [req-d0583e36-91fd-4714-9d31-1207eaafb205 req-9cfe7024-aa98-4811-92a2-91c89fdb7f7a service nova] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Received unexpected event network-vif-plugged-22f727c9-9207-46e6-a471-48de5bf03677 for instance with vm_state deleted and task_state None.
Apr 14 03:24:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:50 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442675.7662914, d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:24:50 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] VM Stopped (Lifecycle Event)
Apr 14 03:24:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-08f3ad00-1fa3-40a9-98ca-3767289aa27b None None] [instance: d7277eb6-c2d1-4c3a-8986-51d3fbf1c29e] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:24:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:24:57 user nova-compute[71323]: INFO nova.compute.claims [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Claim successful on node user
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.251s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:24:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.policy [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c1390cde350341c3ac0275e429146cf8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b689b65111045168cf21c51bf1274f2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:24:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Creating image(s)
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "/opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.129s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk 1073741824" returned: 0 in 0.045s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.180s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking if we can resize image /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Successfully created port: 08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.146s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Cannot resize image /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'migration_context' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Ensure instance console log exists: /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:24:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:24:58 user nova-compute[71323]: INFO nova.compute.manager [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Terminating instance
Apr 14 03:24:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Successfully updated port: 08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquired lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-changed-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Refreshing instance network info cache due to event network-changed-08a928bc-74c8-42cf-92a5-f3470037e540. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] Acquiring lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Instance destroyed successfully.
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lazy-loading 'resources' on Instance uuid 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1553175621',display_name='tempest-VolumesNegativeTest-instance-1553175621',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1553175621',id=40,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:23:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1ccb5ceb9f7641a9b6265de1f3c38c89',ramdisk_id='',reservation_id='r-x4t4rt88',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesNegativeTest-1844013062',owner_user_name='tempest-VolumesNegativeTest-1844013062-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:23:16Z,user_data=None,user_id='85a2f927a9b94b6a8e9d91932bab0af6',uuid=864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converting VIF {"id": "587e1221-a4b5-4e33-8621-2606be640411", "address": "fa:16:3e:99:34:d1", "network": {"id": "17231c8f-ac0c-4a61-92a4-3ecc5cc603af", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.235", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "172ef384b95f49dc8d6e8f73eb7dae92", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap587e1221-a4", "ovs_interfaceid": "587e1221-a4b5-4e33-8621-2606be640411", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG os_vif [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap587e1221-a4, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:59 user nova-compute[71323]: INFO os_vif [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:99:34:d1,bridge_name='br-int',has_traffic_filtering=True,id=587e1221-a4b5-4e33-8621-2606be640411,network=Network(17231c8f-ac0c-4a61-92a4-3ecc5cc603af),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap587e1221-a4')
Apr 14 03:24:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Deleting instance files /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420_del
Apr 14 03:24:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Deletion of /opt/stack/data/nova/instances/864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420_del complete
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating instance_info_cache with network_info: [{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Releasing lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Instance network_info: |[{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] Acquired lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Refreshing network info cache for port 08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Start _get_guest_xml network_info=[{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:24:59 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:24:59 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:24:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2077804000',display_name='tempest-AttachVolumeNegativeTest-server-2077804000',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2077804000',id=42,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFSndsO1c3Z/qbd8lMMRUmZaXIsLfHJCvVNKXkbSK6BNAUu8fZlUrW/Bo5eB9unTY8Vd9653h8+Lb0zpI92t1zd8+Jm7Lui4VceF8r90LX85pQ7OZIoo8WWYkwS7ywbzgQ==',key_name='tempest-keypair-78190679',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-d20t84bl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:24:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=82dc7444-2886-4be4-b565-99e1cb0a70e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.objects.instance [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'pci_devices' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:24:59 user nova-compute[71323]:   <uuid>82dc7444-2886-4be4-b565-99e1cb0a70e4</uuid>
Apr 14 03:24:59 user nova-compute[71323]:   <name>instance-0000002a</name>
Apr 14 03:24:59 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:24:59 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:24:59 user nova-compute[71323]:   <metadata>
Apr 14 03:24:59 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:24:59 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeNegativeTest-server-2077804000</nova:name>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:24:59</nova:creationTime>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:24:59 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:24:59 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:owner>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:user uuid="c1390cde350341c3ac0275e429146cf8">tempest-AttachVolumeNegativeTest-376301777-project-member</nova:user>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:project uuid="3b689b65111045168cf21c51bf1274f2">tempest-AttachVolumeNegativeTest-376301777</nova:project>
Apr 14 03:24:59 user nova-compute[71323]:       </nova:owner>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:24:59 user nova-compute[71323]:       <nova:ports>
Apr 14 03:24:59 user nova-compute[71323]:         <nova:port uuid="08a928bc-74c8-42cf-92a5-f3470037e540">
Apr 14 03:24:59 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Apr 14 03:24:59 user nova-compute[71323]:         </nova:port>
Apr 14 03:24:59 user nova-compute[71323]:       </nova:ports>
Apr 14 03:24:59 user nova-compute[71323]:     </nova:instance>
Apr 14 03:24:59 user nova-compute[71323]:   </metadata>
Apr 14 03:24:59 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:24:59 user nova-compute[71323]:     <system>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="serial">82dc7444-2886-4be4-b565-99e1cb0a70e4</entry>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="uuid">82dc7444-2886-4be4-b565-99e1cb0a70e4</entry>
Apr 14 03:24:59 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:24:59 user nova-compute[71323]:     </system>
Apr 14 03:24:59 user nova-compute[71323]:   </sysinfo>
Apr 14 03:24:59 user nova-compute[71323]:   <os>
Apr 14 03:24:59 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:24:59 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:24:59 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:24:59 user nova-compute[71323]:   </os>
Apr 14 03:24:59 user nova-compute[71323]:   <features>
Apr 14 03:24:59 user nova-compute[71323]:     <acpi/>
Apr 14 03:24:59 user nova-compute[71323]:     <apic/>
Apr 14 03:24:59 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:24:59 user nova-compute[71323]:   </features>
Apr 14 03:24:59 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:24:59 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:24:59 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:24:59 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:24:59 user nova-compute[71323]:   </clock>
Apr 14 03:24:59 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:24:59 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:24:59 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:24:59 user nova-compute[71323]:   </cpu>
Apr 14 03:24:59 user nova-compute[71323]:   <devices>
Apr 14 03:24:59 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:24:59 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:24:59 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk"/>
Apr 14 03:24:59 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:24:59 user nova-compute[71323]:     </disk>
Apr 14 03:24:59 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:24:59 user nova-compute[71323]:       <mac address="fa:16:3e:0a:6a:b2"/>
Apr 14 03:24:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:24:59 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:24:59 user nova-compute[71323]:       <target dev="tap08a928bc-74"/>
Apr 14 03:24:59 user nova-compute[71323]:     </interface>
Apr 14 03:24:59 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:24:59 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/console.log" append="off"/>
Apr 14 03:24:59 user nova-compute[71323]:     </serial>
Apr 14 03:24:59 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:24:59 user nova-compute[71323]:     <video>
Apr 14 03:24:59 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:24:59 user nova-compute[71323]:     </video>
Apr 14 03:24:59 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:24:59 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:24:59 user nova-compute[71323]:     </rng>
Apr 14 03:24:59 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:24:59 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:24:59 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:24:59 user nova-compute[71323]:     </memballoon>
Apr 14 03:24:59 user nova-compute[71323]:   </devices>
Apr 14 03:24:59 user nova-compute[71323]: </domain>
Apr 14 03:24:59 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:24:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2077804000',display_name='tempest-AttachVolumeNegativeTest-server-2077804000',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2077804000',id=42,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFSndsO1c3Z/qbd8lMMRUmZaXIsLfHJCvVNKXkbSK6BNAUu8fZlUrW/Bo5eB9unTY8Vd9653h8+Lb0zpI92t1zd8+Jm7Lui4VceF8r90LX85pQ7OZIoo8WWYkwS7ywbzgQ==',key_name='tempest-keypair-78190679',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-d20t84bl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:24:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=82dc7444-2886-4be4-b565-99e1cb0a70e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG os_vif [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap08a928bc-74, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap08a928bc-74, col_values=(('external_ids', {'iface-id': '08a928bc-74c8-42cf-92a5-f3470037e540', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0a:6a:b2', 'vm-uuid': '82dc7444-2886-4be4-b565-99e1cb0a70e4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:24:59 user nova-compute[71323]: INFO nova.compute.manager [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Took 0.89 seconds to destroy the instance on the hypervisor.
Apr 14 03:24:59 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:24:59 user nova-compute[71323]: INFO os_vif [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74')
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:24:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:0a:6a:b2, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updated VIF entry in instance network info cache for port 08a928bc-74c8-42cf-92a5-f3470037e540. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.network.neutron [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating instance_info_cache with network_info: [{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-948ada83-640e-4a2c-afde-d538812e08d4 req-dd094c60-e047-48aa-8cee-8a726492b602 service nova] Releasing lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:00 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Took 0.46 seconds to deallocate network for instance.
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a3df7ed-b031-45e7-9028-638ad19ab672 req-83b9765d-bca1-44c1-b1e4-57bcdc9ae2d9 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-vif-deleted-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:25:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.145s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:00 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Deleted allocations for instance 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420
Apr 14 03:25:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-cf7715b3-2a5a-4ce9-8b72-d630c0b8f86d tempest-VolumesNegativeTest-1844013062 tempest-VolumesNegativeTest-1844013062-project-member] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.654s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-vif-unplugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] No waiting events found dispatching network-vif-unplugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:01 user nova-compute[71323]: WARNING nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received unexpected event network-vif-unplugged-587e1221-a4b5-4e33-8621-2606be640411 for instance with vm_state deleted and task_state None.
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Acquiring lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] Lock "864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] No waiting events found dispatching network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:01 user nova-compute[71323]: WARNING nova.compute.manager [req-b9bba94c-53e8-4a54-b320-c0e4e88bc009 req-c4477045-bd37-4cd9-aa43-2d6f573e1108 service nova] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Received unexpected event network-vif-plugged-587e1221-a4b5-4e33-8621-2606be640411 for instance with vm_state deleted and task_state None.
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG nova.compute.manager [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] No waiting events found dispatching network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:01 user nova-compute[71323]: WARNING nova.compute.manager [req-d0548eb3-1d5a-4926-b213-cb30d2b98946 req-2e706860-e347-456b-b2ba-fdcdaa19f085 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received unexpected event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 for instance with vm_state building and task_state spawning.
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442703.3508496, 82dc7444-2886-4be4-b565-99e1cb0a70e4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] VM Resumed (Lifecycle Event)
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Instance spawned successfully.
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442703.3519413, 82dc7444-2886-4be4-b565-99e1cb0a70e4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] VM Started (Lifecycle Event)
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Took 5.67 seconds to spawn the instance on the hypervisor.
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:03 user nova-compute[71323]: INFO nova.compute.manager [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Took 6.21 seconds to build instance.
Apr 14 03:25:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-fcec5671-4e82-4837-bf41-df190aaa6068 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.296s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] No waiting events found dispatching network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:03 user nova-compute[71323]: WARNING nova.compute.manager [req-99bc3e83-442b-4e2d-ac15-8445c2ca1d7c req-2199cc62-ae3a-4930-bad8-f7b1f21461d8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received unexpected event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 for instance with vm_state active and task_state None.
Apr 14 03:25:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:08 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:25:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:12 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:25:13 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8499MB free_disk=26.37541961669922GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:25:13 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.215s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442699.7049015, 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:14 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] VM Stopped (Lifecycle Event)
Apr 14 03:25:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-ba46164e-e975-4081-a9b9-83ef074ecb65 None None] [instance: 864cbb2e-a7cb-4cc3-beab-a9b4e1cbe420] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:25:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:25:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.049s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:31 user nova-compute[71323]: INFO nova.compute.manager [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Attaching volume 5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f to /dev/vdb
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.utils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccbfc2c8-0cd0-4d31-8b4c-9880a193b5ea]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fbbf803-fd6e-4913-b3a3-bd50af5ab5e5]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[966a4770-aecf-4837-bd79-f58b92a6bc6f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb4401c1-48dd-4bdb-8f48-7ae862b3957b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:25:32 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:25:32 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[af8c5a38-9b40-4c1c-a271-1bd501f25924]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG os_brick.utils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== get_connector_properties: return (162ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:25:32 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating existing volume attachment record: 5abc96a7-2db7-4fbc-aa97-121b04fc25a5 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:25:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'riVmtpEC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:25:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:25:36 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:25:36 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[61a2128a-5968-44aa-8aa7-8c5e7b4c8b08]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:25:36 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efc8a258-3b4f-4689-b02a-af4b456cc7cd]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f] added
Apr 14 03:25:36 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a96040a-b891-4d79-9b7a-e06aea482546]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b49b5bd5-b97f-4a33-a392-1ecaa330cef5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v riVmtpEC {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v riVmtpEC" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[000b129c-2c9d-43a0-bc9d-1bdcc92973e7]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'riVmtpEC'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1264240c-4e53-45ed-affb-bc7e0d7afbbf]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[80308042-193c-48f9-959d-244d4001ff8b]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:25:37 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:25:37 user nova-compute[71323]:  {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:25:37 user nova-compute[71323]: WARNING os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 03:25:37 user nova-compute[71323]: 
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --login" returned: 0 in 0.046s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1efe695-f3ae-46dd-a00f-ecbd3003b20c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260]
Apr 14 03:25:37 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260] successful.
Apr 14 03:25:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[be59b85e-82e1-4576-982c-8e3d7ba24379]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[89385eb0-5403-409f-9589-afb080d802b2]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:25:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:25:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.267s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] HCTL ('33', '-', '-', 0) found on session 49 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[758b0418-9ceb-4436-94f7-0ed88e1f619c]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Searching for a device in session 49 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'riVmtpEC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.300s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== connect_volume: return (1301ms) {'type': 'block', 'scsi_wwn': '26461326463616362', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26461326463616362', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.objects.instance [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:25:38 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:25:38 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:25:38 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:25:38 user nova-compute[71323]:   <serial>5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f</serial>
Apr 14 03:25:38 user nova-compute[71323]: </disk>
Apr 14 03:25:38 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] No VIF found with MAC fa:16:3e:94:e8:44, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:25:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-038a3209-0938-49e7-b4cd-ec37655b430a tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.812s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:25:40 user nova-compute[71323]: INFO nova.compute.claims [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Claim successful on node user
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.338s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:25:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.policy [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f88237b825b24effad17900bbf645ab8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ed26668c2cb49c38d16ff728d38bef2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:25:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Creating image(s)
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:40 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk 1073741824" returned: 0 in 0.049s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.188s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Checking if we can resize image /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing instance network info cache due to event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.network.neutron [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Cannot resize image /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'migration_context' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Ensure instance console log exists: /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:41 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Successfully created port: b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.network.neutron [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated VIF entry in instance network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.network.neutron [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.254", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3ea9952f-212e-4ada-a1fd-45d56fbe3118 req-3b578bf7-c393-4e06-b95f-9d14be9fefe8 service nova] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Successfully updated port: b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing instance network info cache due to event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:25:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.neutron [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance network_info: |[{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start _get_guest_xml network_info=[{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:25:43 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:25:43 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJClBvq+izruhg+hEGBNF3dbytQyMHTMScVlN0JyjMZhWJs7Q0qW6oRUDMkWmHhWGirQ5OTk6+cGFOrE1Qs+cgpleFl1Ec3q//B2mJ5NTiLySOLz8iea1keakGG6m35P8g==',key_name='tempest-keypair-1931660222',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:25:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_devices' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:25:43 user nova-compute[71323]:   <uuid>16e9b236-ce8f-4231-af55-e694bc43dde4</uuid>
Apr 14 03:25:43 user nova-compute[71323]:   <name>instance-0000002b</name>
Apr 14 03:25:43 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:25:43 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:25:43 user nova-compute[71323]:   <metadata>
Apr 14 03:25:43 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:25:43 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-240447376</nova:name>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:25:43</nova:creationTime>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:25:43 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:25:43 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:owner>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:user uuid="f88237b825b24effad17900bbf645ab8">tempest-AttachVolumeShelveTestJSON-1546459049-project-member</nova:user>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:project uuid="7ed26668c2cb49c38d16ff728d38bef2">tempest-AttachVolumeShelveTestJSON-1546459049</nova:project>
Apr 14 03:25:43 user nova-compute[71323]:       </nova:owner>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:25:43 user nova-compute[71323]:       <nova:ports>
Apr 14 03:25:43 user nova-compute[71323]:         <nova:port uuid="b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4">
Apr 14 03:25:43 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 03:25:43 user nova-compute[71323]:         </nova:port>
Apr 14 03:25:43 user nova-compute[71323]:       </nova:ports>
Apr 14 03:25:43 user nova-compute[71323]:     </nova:instance>
Apr 14 03:25:43 user nova-compute[71323]:   </metadata>
Apr 14 03:25:43 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:25:43 user nova-compute[71323]:     <system>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="serial">16e9b236-ce8f-4231-af55-e694bc43dde4</entry>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="uuid">16e9b236-ce8f-4231-af55-e694bc43dde4</entry>
Apr 14 03:25:43 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:25:43 user nova-compute[71323]:     </system>
Apr 14 03:25:43 user nova-compute[71323]:   </sysinfo>
Apr 14 03:25:43 user nova-compute[71323]:   <os>
Apr 14 03:25:43 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:25:43 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:25:43 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:25:43 user nova-compute[71323]:   </os>
Apr 14 03:25:43 user nova-compute[71323]:   <features>
Apr 14 03:25:43 user nova-compute[71323]:     <acpi/>
Apr 14 03:25:43 user nova-compute[71323]:     <apic/>
Apr 14 03:25:43 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:25:43 user nova-compute[71323]:   </features>
Apr 14 03:25:43 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:25:43 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:25:43 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:25:43 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:25:43 user nova-compute[71323]:   </clock>
Apr 14 03:25:43 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:25:43 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:25:43 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:25:43 user nova-compute[71323]:   </cpu>
Apr 14 03:25:43 user nova-compute[71323]:   <devices>
Apr 14 03:25:43 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:25:43 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:25:43 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk"/>
Apr 14 03:25:43 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:25:43 user nova-compute[71323]:     </disk>
Apr 14 03:25:43 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:25:43 user nova-compute[71323]:       <mac address="fa:16:3e:11:d8:7b"/>
Apr 14 03:25:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:25:43 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:25:43 user nova-compute[71323]:       <target dev="tapb47d2f30-8e"/>
Apr 14 03:25:43 user nova-compute[71323]:     </interface>
Apr 14 03:25:43 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:25:43 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/console.log" append="off"/>
Apr 14 03:25:43 user nova-compute[71323]:     </serial>
Apr 14 03:25:43 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:25:43 user nova-compute[71323]:     <video>
Apr 14 03:25:43 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:25:43 user nova-compute[71323]:     </video>
Apr 14 03:25:43 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:25:43 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:25:43 user nova-compute[71323]:     </rng>
Apr 14 03:25:43 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:25:43 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:25:43 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:25:43 user nova-compute[71323]:     </memballoon>
Apr 14 03:25:43 user nova-compute[71323]:   </devices>
Apr 14 03:25:43 user nova-compute[71323]: </domain>
Apr 14 03:25:43 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJClBvq+izruhg+hEGBNF3dbytQyMHTMScVlN0JyjMZhWJs7Q0qW6oRUDMkWmHhWGirQ5OTk6+cGFOrE1Qs+cgpleFl1Ec3q//B2mJ5NTiLySOLz8iea1keakGG6m35P8g==',key_name='tempest-keypair-1931660222',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:25:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG os_vif [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb47d2f30-8e, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb47d2f30-8e, col_values=(('external_ids', {'iface-id': 'b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:11:d8:7b', 'vm-uuid': '16e9b236-ce8f-4231-af55-e694bc43dde4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:43 user nova-compute[71323]: INFO os_vif [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e')
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No VIF found with MAC fa:16:3e:11:d8:7b, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updated VIF entry in instance network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG nova.network.neutron [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be9e1157-0342-4d10-baff-34a52461cb08 req-18a02b56-3a51-42c7-8d39-8535900d4151 service nova] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:44 user nova-compute[71323]: INFO nova.compute.manager [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Rebooting instance
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.compute.manager [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing instance network info cache due to event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.254", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.network.neutron [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:25:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG nova.network.neutron [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated VIF entry in instance network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG nova.network.neutron [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.254", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:25:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9b42c355-74e3-415a-a3bb-19e6750af7d1 req-573fac55-f586-42c8-8be4-a76b39bb9c13 service nova] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state building and task_state spawning.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:46 user nova-compute[71323]: WARNING nova.compute.manager [req-8f43f8da-1e59-4512-93f3-32122d6fd11f req-24bb8a65-d602-4196-93e2-8ac4fc182523 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state building and task_state spawning.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442746.7706754, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Resumed (Lifecycle Event)
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance spawned successfully.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442746.7721686, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Started (Lifecycle Event)
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Took 5.95 seconds to spawn the instance on the hypervisor.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:46 user nova-compute[71323]: INFO nova.compute.manager [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Took 6.59 seconds to build instance.
Apr 14 03:25:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a521d73c-35f0-4629-8e76-62843dc2ec14 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.686s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:47 user nova-compute[71323]: WARNING nova.compute.manager [req-6d6dc222-0a6b-40f9-ba68-a29e5f9a18d6 req-8afb9e6b-2764-4ddc-a927-04e8e5dc00af service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state active and task_state reboot_started.
Apr 14 03:25:48 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance shutdown successfully.
Apr 14 03:25:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:49 user nova-compute[71323]: WARNING nova.compute.manager [req-e41a695a-eb12-413a-a406-a14d90569321 req-d3a2579e-1b29-4f59-8058-140b70dce339 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state active and task_state reboot_started.
Apr 14 03:25:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.host [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Removed pending event for f8042c4c-0ff6-4533-8f59-de5c41ede33f due to event {{(pid=71323) _event_emit_delayed /opt/stack/nova/nova/virt/libvirt/host.py:438}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442751.7511678, f8042c4c-0ff6-4533-8f59-de5c41ede33f => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:51 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] VM Resumed (Lifecycle Event)
Apr 14 03:25:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance running successfully.
Apr 14 03:25:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance soft rebooted successfully.
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:51 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] During sync_power_state the instance has a pending task (reboot_started). Skip.
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442751.7546782, f8042c4c-0ff6-4533-8f59-de5c41ede33f => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:25:51 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] VM Started (Lifecycle Event)
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: reboot_started, current DB power_state: 1, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:25:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-83d8ce35-c4c5-4731-90c4-15afa33f2aa1 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 7.578s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:52 user nova-compute[71323]: WARNING nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state active and task_state reboot_started.
Apr 14 03:25:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:25:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:25:52 user nova-compute[71323]: WARNING nova.compute.manager [req-6a05f9fe-2603-4a38-b633-0a22dede53d1 req-a877f54d-1124-4723-b38b-b7f845778c40 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state active and task_state reboot_started.
Apr 14 03:25:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:25:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:26:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:26:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:26:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8403MB free_disk=26.32297134399414GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 16e9b236-ce8f-4231-af55-e694bc43dde4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:26:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.237s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:26:16 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:26:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.254", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:26:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:26:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:26:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:26:20 user nova-compute[71323]: INFO nova.compute.claims [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Claim successful on node user
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.236s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:26:20 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:26:20 user nova-compute[71323]: INFO nova.virt.block_device [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Booting with volume 3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 at /dev/vda
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.policy [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '61f477304a044148ade1c6acffd30712', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e0979cf1834435bb0007899da52e863', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c53102f-dd76-4dfb-bc64-19ca9af6281f]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b81092ca-90d6-4314-8c1e-e4fa75675573]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[117b34f8-9cae-40f4-a733-662e4c99e385]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd9eadc7-393e-4a8f-b281-ca616d386308]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:26:20 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:26:20 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[95d6d13e-b930-47ab-aee0-09bbbbc5499a]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG os_brick.utils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:26:20 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating existing volume attachment record: 857453f9-dc5f-4e5f-8899-e783d463ef89 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:26:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Successfully created port: a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Successfully updated port: a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-changed-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.compute.manager [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing instance network info cache due to event network-changed-a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance network_info: |[{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:26:22 user nova-compute[71323]: DEBUG nova.network.neutron [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing network info cache for port a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:26:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:23 user nova-compute[71323]: DEBUG nova.network.neutron [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated VIF entry in instance network info cache for port a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:26:23 user nova-compute[71323]: DEBUG nova.network.neutron [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:26:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-480f078a-0545-4b25-927f-d0909c345791 req-50723075-77a4-407e-9bd6-e7138bfba36d service nova] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:26:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:26:25 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Creating image(s)
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Ensure instance console log exists: /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start _get_guest_xml network_info=[{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '857453f9-dc5f-4e5f-8899-e783d463ef89', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ce889d46-fac9-4f77-9d66-6e7753d8a0e0', 'attached_at': '', 'detached_at': '', 'volume_id': '3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'serial': '3a8e1f1a-36b6-47b7-95f2-d31cd1b55808'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:26:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:26:25 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:26:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:26:25 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:26:25 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ae6b122-cffa-41d3-a41b-47cbf0e88590]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:26:25 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[84c8aaa2-f7e9-4091-8c39-e70d5084046a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808] added
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0c6793b-35f0-44d6-a58f-d8799bae2b3e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ceb43c55-5356-4562-8cfc-90a05f96d521]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TcJSZhIU {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TcJSZhIU" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba8db430-bce0-4118-af21-8baf27b827be]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'TcJSZhIU'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[720398db-7409-4a5e-8dcf-1ba4dd6a6137]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5b10d91-eaa9-4734-90fe-58d2743190a4]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e03e74be-4cb9-49d0-a37e-4fb5702f8f22]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]
Apr 14 03:26:25 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebdc8130-a318-40a3-b0e9-57cca5af0071]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b45178f-ecef-4f3b-8d8b-0c68c4b8e232]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:25 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:25 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:25 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.256s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] HCTL ('34', '-', '-', 0) found on session 50 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aad9c655-ec32-44b0-b995-7fcc6a219393]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:25 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Searching for a device in session 50 and hctl ['34', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Searching for a device in session 50 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.283s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== connect_volume: return (1284ms) {'type': 'block', 'scsi_wwn': '23264633231623239', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23264633231623239', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFaRzDIjymQDKgKuJGVKrY9YoDm6sxEMtltnJTNSG+jb5Yzbpdqb24ukF2QSYjCu5ZtFM9xpqKafHtv/J2C+itYCqUfzijhnmtqKn1C/3bTReImbLmTyuHqlHmJ8FHo8QQ==',key_name='tempest-TestShelveInstance-472203162',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:26:21Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'pci_devices' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:26:26 user nova-compute[71323]:   <uuid>ce889d46-fac9-4f77-9d66-6e7753d8a0e0</uuid>
Apr 14 03:26:26 user nova-compute[71323]:   <name>instance-0000002c</name>
Apr 14 03:26:26 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:26:26 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:26:26 user nova-compute[71323]:   <metadata>
Apr 14 03:26:26 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:26:26 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:26:26 user nova-compute[71323]:       <nova:name>tempest-TestShelveInstance-server-1348709575</nova:name>
Apr 14 03:26:26 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:26:25</nova:creationTime>
Apr 14 03:26:26 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:26:26 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:26:26 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:26:26 user nova-compute[71323]:       <nova:owner>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:user uuid="61f477304a044148ade1c6acffd30712">tempest-TestShelveInstance-1966208901-project-member</nova:user>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:project uuid="1e0979cf1834435bb0007899da52e863">tempest-TestShelveInstance-1966208901</nova:project>
Apr 14 03:26:26 user nova-compute[71323]:       </nova:owner>
Apr 14 03:26:26 user nova-compute[71323]:       <nova:ports>
Apr 14 03:26:26 user nova-compute[71323]:         <nova:port uuid="a0610502-2adf-475c-822c-d96d457872a8">
Apr 14 03:26:26 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 03:26:26 user nova-compute[71323]:         </nova:port>
Apr 14 03:26:26 user nova-compute[71323]:       </nova:ports>
Apr 14 03:26:26 user nova-compute[71323]:     </nova:instance>
Apr 14 03:26:26 user nova-compute[71323]:   </metadata>
Apr 14 03:26:26 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:26:26 user nova-compute[71323]:     <system>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="serial">ce889d46-fac9-4f77-9d66-6e7753d8a0e0</entry>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="uuid">ce889d46-fac9-4f77-9d66-6e7753d8a0e0</entry>
Apr 14 03:26:26 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:26:26 user nova-compute[71323]:     </system>
Apr 14 03:26:26 user nova-compute[71323]:   </sysinfo>
Apr 14 03:26:26 user nova-compute[71323]:   <os>
Apr 14 03:26:26 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:26:26 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:26:26 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:26:26 user nova-compute[71323]:   </os>
Apr 14 03:26:26 user nova-compute[71323]:   <features>
Apr 14 03:26:26 user nova-compute[71323]:     <acpi/>
Apr 14 03:26:26 user nova-compute[71323]:     <apic/>
Apr 14 03:26:26 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:26:26 user nova-compute[71323]:   </features>
Apr 14 03:26:26 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:26:26 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:26:26 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:26:26 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:26:26 user nova-compute[71323]:   </clock>
Apr 14 03:26:26 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:26:26 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:26:26 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:26:26 user nova-compute[71323]:   </cpu>
Apr 14 03:26:26 user nova-compute[71323]:   <devices>
Apr 14 03:26:26 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:26:26 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:26:26 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:26:26 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:26:26 user nova-compute[71323]:       <serial>3a8e1f1a-36b6-47b7-95f2-d31cd1b55808</serial>
Apr 14 03:26:26 user nova-compute[71323]:     </disk>
Apr 14 03:26:26 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:26:26 user nova-compute[71323]:       <mac address="fa:16:3e:02:35:6f"/>
Apr 14 03:26:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:26:26 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:26:26 user nova-compute[71323]:       <target dev="tapa0610502-2a"/>
Apr 14 03:26:26 user nova-compute[71323]:     </interface>
Apr 14 03:26:26 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:26:26 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0/console.log" append="off"/>
Apr 14 03:26:26 user nova-compute[71323]:     </serial>
Apr 14 03:26:26 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:26:26 user nova-compute[71323]:     <video>
Apr 14 03:26:26 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:26:26 user nova-compute[71323]:     </video>
Apr 14 03:26:26 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:26:26 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:26:26 user nova-compute[71323]:     </rng>
Apr 14 03:26:26 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:26:26 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:26:26 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:26:26 user nova-compute[71323]:     </memballoon>
Apr 14 03:26:26 user nova-compute[71323]:   </devices>
Apr 14 03:26:26 user nova-compute[71323]: </domain>
Apr 14 03:26:26 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFaRzDIjymQDKgKuJGVKrY9YoDm6sxEMtltnJTNSG+jb5Yzbpdqb24ukF2QSYjCu5ZtFM9xpqKafHtv/J2C+itYCqUfzijhnmtqKn1C/3bTReImbLmTyuHqlHmJ8FHo8QQ==',key_name='tempest-TestShelveInstance-472203162',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:26:21Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG os_vif [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa0610502-2a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa0610502-2a, col_values=(('external_ids', {'iface-id': 'a0610502-2adf-475c-822c-d96d457872a8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:02:35:6f', 'vm-uuid': 'ce889d46-fac9-4f77-9d66-6e7753d8a0e0'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:26 user nova-compute[71323]: INFO os_vif [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a')
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:26:26 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] No VIF found with MAC fa:16:3e:02:35:6f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:26:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG nova.compute.manager [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:26:28 user nova-compute[71323]: WARNING nova.compute.manager [req-440a7f59-e563-41ec-8ab8-47def873e19e req-e6182649-758d-43ac-8b88-984601902ed7 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state building and task_state spawning.
Apr 14 03:26:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442790.073883, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Resumed (Lifecycle Event)
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance spawned successfully.
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442790.0760891, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Started (Lifecycle Event)
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Took 4.85 seconds to spawn the instance on the hypervisor.
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:30 user nova-compute[71323]: DEBUG nova.compute.manager [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:26:30 user nova-compute[71323]: WARNING nova.compute.manager [req-27649f39-a149-4e4d-bcc2-7ac82740c9fe req-752e3467-1f78-433c-9369-580be0f87add service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state building and task_state spawning.
Apr 14 03:26:30 user nova-compute[71323]: INFO nova.compute.manager [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Took 9.94 seconds to build instance.
Apr 14 03:26:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3d42bc61-7184-4f03-addb-02d10e4a0ce6 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.039s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-changed-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:26:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Refreshing instance network info cache due to event network-changed-08a928bc-74c8-42cf-92a5-f3470037e540. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:26:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] Acquiring lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:26:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] Acquired lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:26:48 user nova-compute[71323]: DEBUG nova.network.neutron [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Refreshing network info cache for port 08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:26:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updated VIF entry in instance network info cache for port 08a928bc-74c8-42cf-92a5-f3470037e540. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:26:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating instance_info_cache with network_info: [{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:26:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d5332d43-8ce1-49de-8e5b-0d9662260a61 req-69a0dc79-a6a6-43c6-a928-1e336f2c49c3 service nova] Releasing lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:26:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG nova.objects.instance [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.324s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:51 user nova-compute[71323]: INFO nova.compute.manager [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Attaching volume 7ef98f79-a820-44e4-bde2-010a709dc93c to /dev/vdb
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.utils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fb30d0d-8c31-43b8-bbe0-7c9f418b5c4d]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[15056f79-e445-4cba-a646-4b8c51919038]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec04e3d7-fa0f-4e78-9424-eda1cce2fb22]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[be655bd5-3d1a-487f-bb96-af48df3d451a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:26:51 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:26:51 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ca64887-5883-47d3-8df1-f70623b1c06f]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG os_brick.utils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== get_connector_properties: return (112ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:26:51 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating existing volume attachment record: cd271231-6906-48c9-a478-0306a67b7719 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:26:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EyFqhPUm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:26:56 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:26:56 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:26:56 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb507a34-9a58-4e7d-98e9-3c39a646c889]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:26:56 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fb6e745-2ff1-420d-9736-f83edc12b0a1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c] added
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.032s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[67eabe01-37e4-4f44-8677-00cf21b2bff5]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bfe3556-c433-4bc5-ab4b-a4effae51ce1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EyFqhPUm {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v EyFqhPUm" returned: 0 in 0.027s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[164ebfb6-4303-4614-acab-d8a2236d23dd]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'EyFqhPUm'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfb5ee4d-ab6a-4ce1-9505-48dc2f6c842d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7c27a66-223c-41a2-8ffd-4b7296314149]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b16c6cf-7897-4a45-a87c-fee2aa04f1ca]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260]
Apr 14 03:26:56 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260] successful.
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b85b02f0-cd0b-48dc-b9a1-9635a70d54e2]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6494bd6d-f641-4d5f-b844-589077dee772]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:56 user nova-compute[71323]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)
Apr 14 03:26:56 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.288s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] HCTL ('35', '-', '-', 0) found on session 51 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[86e8d5c1-28ce-4e16-ac36-69fe06170b71]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 51 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:26:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 51 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EyFqhPUm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.317s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== connect_volume: return (1319ms) {'type': 'block', 'scsi_wwn': '26532393537333930', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26532393537333930', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.objects.instance [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:26:57 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:26:57 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:26:57 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:26:57 user nova-compute[71323]:   <serial>7ef98f79-a820-44e4-bde2-010a709dc93c</serial>
Apr 14 03:26:57 user nova-compute[71323]: </disk>
Apr 14 03:26:57 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:26:57 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:0a:6a:b2, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:26:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-55017929-b33b-45bd-81cb-0fe8c6267b2d tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.261s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:26:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:26:58 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:26:58 user nova-compute[71323]: INFO nova.compute.manager [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Detaching volume 7ef98f79-a820-44e4-bde2-010a709dc93c
Apr 14 03:26:58 user nova-compute[71323]: INFO nova.virt.block_device [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Attempting to driver detach volume 7ef98f79-a820-44e4-bde2-010a709dc93c from mountpoint /dev/vdb
Apr 14 03:26:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Attempting to detach device vdb from instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:26:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:26:58 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:26:58 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:26:58 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:26:58 user nova-compute[71323]:   <serial>7ef98f79-a820-44e4-bde2-010a709dc93c</serial>
Apr 14 03:26:58 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:26:58 user nova-compute[71323]: </disk>
Apr 14 03:26:58 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:26:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 from the persistent domain config.
Apr 14 03:26:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:26:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:26:58 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:26:58 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:26:58 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:26:58 user nova-compute[71323]:   <serial>7ef98f79-a820-44e4-bde2-010a709dc93c</serial>
Apr 14 03:26:58 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:26:58 user nova-compute[71323]: </disk>
Apr 14 03:26:58 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442819.1307495, 82dc7444-2886-4be4-b565-99e1cb0a70e4 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:26:59 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 from the live domain config.
Apr 14 03:26:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'EyFqhPUm', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:26:59 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e60e4db6-331d-4fcc-bc40-bfb0d07c8b80]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:26:59 user nova-compute[71323]: No targets found.
Apr 14 03:26:59 user nova-compute[71323]: iSNS:
Apr 14 03:26:59 user nova-compute[71323]: No targets found.
Apr 14 03:26:59 user nova-compute[71323]: STATIC:
Apr 14 03:26:59 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:26:59 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:26:59 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:26:59 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c
Apr 14 03:26:59 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:26:59 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:26:59 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f
Apr 14 03:26:59 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:26:59 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:26:59 user nova-compute[71323]: FIRMWARE:
Apr 14 03:26:59 user nova-compute[71323]: No targets found.
Apr 14 03:26:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:26:59 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:26:59 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[20b03079-8390-44cd-9e42-05c46a0d9a9c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0b1f85c-b8b7-4a90-9241-9d77442be0ac]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:59 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:59 user nova-compute[71323]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)
Apr 14 03:26:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:26:59 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:26:59 user nova-compute[71323]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c (non-flash)
Apr 14 03:26:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cdca290>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f46c9e7-4296-4ecb-864e-cc484a6b14d3]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b43c4471-d550-4fe5-8b66-a0a48cec128d]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d2f2c39-b5be-4bd4-8e88-d41a9869c845]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.029s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbe89340-a1da-4145-a0a5-4ad6d67bbb0c]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a17a0c9-0acc-4047-921e-e1b28c10662a]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d7ab0da-807b-4510-9d9f-035cabf1dc67]: (4, ('Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260]\nLogout of [sid: 51, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260]
Apr 14 03:26:59 user nova-compute[71323]: Logout of [sid: 51, target: iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c, portal: 172.16.0.220,3260] successful.
Apr 14 03:26:59 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7ef98f79-a820-44e4-bde2-010a709dc93c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[20943c50-6994-495f-9074-12625b6ac6a6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.305s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== disconnect_volume: return (307ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:26:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:00 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:27:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e18747fa-f296-4ff7-bc91-6b6c2779f168 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.635s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:27:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:27:13 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.131s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4/disk --force-share --output=json" returned: 0 in 0.140s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:27:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:27:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8328MB free_disk=26.32046890258789GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 82dc7444-2886-4be4-b565-99e1cb0a70e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 16e9b236-ce8f-4231-af55-e694bc43dde4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=1024MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:27:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.296s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:17 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:27:18 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating instance_info_cache with network_info: [{"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-82dc7444-2886-4be4-b565-99e1cb0a70e4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:27:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:27:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:27:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG nova.compute.manager [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing instance network info cache due to event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updated VIF entry in instance network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG nova.network.neutron [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4d3d05aa-1adb-42b0-ac40-2c896ccfbff8 req-ca8bcef3-8542-463a-abb4-ea54ed94eb3a service nova] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:27:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:34 user nova-compute[71323]: INFO nova.compute.manager [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Shelving
Apr 14 03:27:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:27:35 user nova-compute[71323]: WARNING nova.compute.manager [req-efa17adf-e46f-4b29-9948-da160d9f8c6a req-a9283ce2-ea0b-478f-a87d-55fe461025d9 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state active and task_state shelving.
Apr 14 03:27:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing instance network info cache due to event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:27:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance destroyed successfully.
Apr 14 03:27:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Beginning cold snapshot process
Apr 14 03:27:35 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:27:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk /opt/stack/data/nova/instances/snapshots/tmpr9udl1ia/61ed6c3677924a09901bb4f5a804eafb {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated VIF entry in instance network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.network.neutron [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.254", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-42db0955-d44f-42ca-ae65-b777f9524ffd req-2f7c8b47-b884-4a7c-a3ea-bc47874a87a1 service nova] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk /opt/stack/data/nova/instances/snapshots/tmpr9udl1ia/61ed6c3677924a09901bb4f5a804eafb" returned: 0 in 0.449s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Snapshot extracted, beginning image upload
Apr 14 03:27:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:36 user nova-compute[71323]: INFO nova.compute.manager [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Detaching volume 5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f
Apr 14 03:27:36 user nova-compute[71323]: INFO nova.virt.block_device [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Attempting to driver detach volume 5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f from mountpoint /dev/vdb
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Attempting to detach device vdb from instance f8042c4c-0ff6-4533-8f59-de5c41ede33f from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:27:36 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:27:36 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:27:36 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:27:36 user nova-compute[71323]:   <serial>5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f</serial>
Apr 14 03:27:36 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:27:36 user nova-compute[71323]: </disk>
Apr 14 03:27:36 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:27:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully detached device vdb from instance f8042c4c-0ff6-4533-8f59-de5c41ede33f from the persistent domain config.
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f8042c4c-0ff6-4533-8f59-de5c41ede33f from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:27:36 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:27:36 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:27:36 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:27:36 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:27:36 user nova-compute[71323]:   <serial>5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f</serial>
Apr 14 03:27:36 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:27:36 user nova-compute[71323]: </disk>
Apr 14 03:27:36 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:27:37 user nova-compute[71323]: WARNING nova.compute.manager [req-e06c30c1-2cde-4449-bc1c-03b30a9a0c92 req-8737b6cb-9513-43da-89d5-4b77cf2b8a20 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state active and task_state shelving_image_uploading.
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681442857.3917494, f8042c4c-0ff6-4533-8f59-de5c41ede33f => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:27:37 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully detached device vdb from instance f8042c4c-0ff6-4533-8f59-de5c41ede33f from the live domain config.
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'riVmtpEC', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:27:37 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8841dc9-a09a-4924-86f3-1c620b664ab6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:27:37 user nova-compute[71323]: No targets found.
Apr 14 03:27:37 user nova-compute[71323]: iSNS:
Apr 14 03:27:37 user nova-compute[71323]: No targets found.
Apr 14 03:27:37 user nova-compute[71323]: STATIC:
Apr 14 03:27:37 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:27:37 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:27:37 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:27:37 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f
Apr 14 03:27:37 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:27:37 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:27:37 user nova-compute[71323]: FIRMWARE:
Apr 14 03:27:37 user nova-compute[71323]: No targets found.
Apr 14 03:27:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:27:37 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:27:37 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[aafb6f0b-ac5c-4850-864f-0038766bc451]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f12c525-3696-497e-a835-4bae7f0f2328]: (4, ('tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:27:37 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:27:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsi session list stdout=tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f (non-flash)
Apr 14 03:27:37 user nova-compute[71323]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:27:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cda1a20>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8908d18e-e4cb-46b1-805e-d52e75211f0a]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b644af75-3d82-4511-9d86-b8d43b8ce4ad]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1cdd473-c34e-4ae2-a442-d1f05b98fd35]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fc4f97e-e531-4519-aa2d-281633f58294]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.047s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[befe7d90-1a1e-44f1-8ba5-b849855ac39c]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c9c06de-de21-4bff-86b3-4f2719c0de2b]: (4, ('Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260]\nLogout of [sid: 49, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260]
Apr 14 03:27:37 user nova-compute[71323]: Logout of [sid: 49, target: iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f, portal: 172.16.0.220,3260] successful.
Apr 14 03:27:37 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing instance network info cache due to event network-changed-8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] Acquiring lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] Acquired lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.network.neutron [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Refreshing network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b83278d-f8ec-4fad-95e2-f1e5f3e7f27f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b2893f7-2dee-4fa4-9c7d-34a3f78702c3]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.327s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] <== disconnect_volume: return (328ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:27:37 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.network.neutron [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updated VIF entry in instance network info cache for port 8a4c167f-3166-4f90-91c9-1a1d5be3c92d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.network.neutron [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [{"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9d1cb39e-57e1-4c8f-bb47-8a7747f36c9a req-3d3ce9a7-ffa4-46b2-a271-a0852e7588a9 service nova] Releasing lock "refresh_cache-f8042c4c-0ff6-4533-8f59-de5c41ede33f" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.objects.instance [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'flavor' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:27:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Snapshot image upload complete
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-85284b7d-4b51-487d-847b-27ece93a8bc6 tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.711s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:38 user nova-compute[71323]: INFO nova.compute.manager [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Shelve offloading
Apr 14 03:27:38 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance destroyed successfully.
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.compute.manager [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:38 user nova-compute[71323]: DEBUG nova.network.neutron [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.network.neutron [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance destroyed successfully.
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.objects.instance [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'resources' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJClBvq+izruhg+hEGBNF3dbytQyMHTMScVlN0JyjMZhWJs7Q0qW6oRUDMkWmHhWGirQ5OTk6+cGFOrE1Qs+cgpleFl1Ec3q//B2mJ5NTiLySOLz8iea1keakGG6m35P8g==',key_name='tempest-keypair-1931660222',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:25:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:27:38.542369',shelved_host='user',shelved_image_id='ea29233a-8154-40d5-89bf-d522c0d1c520'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:27:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG os_vif [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb47d2f30-8e, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.compute.manager [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing instance network info cache due to event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG nova.network.neutron [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:27:39 user nova-compute[71323]: INFO os_vif [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e')
Apr 14 03:27:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Deleting instance files /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4_del
Apr 14 03:27:39 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Deletion of /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4_del complete
Apr 14 03:27:39 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Deleted allocations for instance 16e9b236-ce8f-4231-af55-e694bc43dde4
Apr 14 03:27:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'numa_topology' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.184s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-122209b6-d57e-4c3d-8ab0-5ff9cbe1bdeb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.365s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updated VIF entry in instance network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG nova.network.neutron [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4e895d41-877e-4b0a-9d97-9d1faede872a req-dbfaacbb-bd58-4d59-90ae-ba33d53aee40 service nova] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:27:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:48 user nova-compute[71323]: INFO nova.compute.manager [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Terminating instance
Apr 14 03:27:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-unplugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] No waiting events found dispatching network-vif-unplugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-03becbab-d694-42bd-957d-c8831ecb8253 req-221844f9-a1b9-4c1b-86fd-1d9fafe9f1c8 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-unplugged-08a928bc-74c8-42cf-92a5-f3470037e540 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Instance destroyed successfully.
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'resources' on Instance uuid 82dc7444-2886-4be4-b565-99e1cb0a70e4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:24:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-2077804000',display_name='tempest-AttachVolumeNegativeTest-server-2077804000',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-2077804000',id=42,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFSndsO1c3Z/qbd8lMMRUmZaXIsLfHJCvVNKXkbSK6BNAUu8fZlUrW/Bo5eB9unTY8Vd9653h8+Lb0zpI92t1zd8+Jm7Lui4VceF8r90LX85pQ7OZIoo8WWYkwS7ywbzgQ==',key_name='tempest-keypair-78190679',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:25:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-d20t84bl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:25:04Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=82dc7444-2886-4be4-b565-99e1cb0a70e4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "08a928bc-74c8-42cf-92a5-f3470037e540", "address": "fa:16:3e:0a:6a:b2", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap08a928bc-74", "ovs_interfaceid": "08a928bc-74c8-42cf-92a5-f3470037e540", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG os_vif [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap08a928bc-74, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:27:49 user nova-compute[71323]: INFO os_vif [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:6a:b2,bridge_name='br-int',has_traffic_filtering=True,id=08a928bc-74c8-42cf-92a5-f3470037e540,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap08a928bc-74')
Apr 14 03:27:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Deleting instance files /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4_del
Apr 14 03:27:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Deletion of /opt/stack/data/nova/instances/82dc7444-2886-4be4-b565-99e1cb0a70e4_del complete
Apr 14 03:27:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:49 user nova-compute[71323]: INFO nova.compute.manager [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Took 0.84 seconds to destroy the instance on the hypervisor.
Apr 14 03:27:49 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:27:49 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:27:50 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Took 0.69 seconds to deallocate network for instance.
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-8bdc9a98-aee4-420e-9e3c-3211991a3537 req-5d421647-7d74-4d78-ac6a-54eaf9c32534 service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-deleted-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442855.6464195, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:27:50 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Stopped (Lifecycle Event)
Apr 14 03:27:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-f6630c37-8da4-45d0-9a0b-454b6ad33338 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:27:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.155s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:50 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Deleted allocations for instance 82dc7444-2886-4be4-b565-99e1cb0a70e4
Apr 14 03:27:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c1f9d874-08df-4ac8-98b2-7057eb498a9e tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.853s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:27:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] Acquiring lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:27:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:27:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] Lock "82dc7444-2886-4be4-b565-99e1cb0a70e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:27:51 user nova-compute[71323]: DEBUG nova.compute.manager [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] No waiting events found dispatching network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:27:51 user nova-compute[71323]: WARNING nova.compute.manager [req-7d9f0a9d-adb9-4b42-8adb-0f3470aaef18 req-54d90ffc-6cc6-411d-a5e1-86affde5699f service nova] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Received unexpected event network-vif-plugged-08a928bc-74c8-42cf-92a5-f3470037e540 for instance with vm_state deleted and task_state None.
Apr 14 03:27:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:27:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:04 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442869.6781812, 82dc7444-2886-4be4-b565-99e1cb0a70e4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:28:04 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] VM Stopped (Lifecycle Event)
Apr 14 03:28:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0ca1da53-ca76-4661-9013-d073e9237e32 None None] [instance: 82dc7444-2886-4be4-b565-99e1cb0a70e4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:08 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-changed-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing instance network info cache due to event network-changed-a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing network info cache for port a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" acquired by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:12 user nova-compute[71323]: INFO nova.compute.manager [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Shelve offloading
Apr 14 03:28:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.compute.manager [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:12 user nova-compute[71323]: WARNING nova.compute.manager [req-5e74a5ad-80f7-4b83-af62-abd74bca04b1 req-baabc713-58a9-44be-9ee4-9ff2c5d53eef service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state active and task_state shelving.
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated VIF entry in instance network info cache for port a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:28:12 user nova-compute[71323]: DEBUG nova.network.neutron [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b61199a9-3f4d-4d4e-bb85-783c1e5fba9b req-ee010787-2ae6-4519-91c8-765b4e2f78db service nova] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:28:13 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance destroyed successfully.
Apr 14 03:28:13 user nova-compute[71323]: DEBUG nova.compute.manager [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG nova.network.neutron [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:28:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance destroyed successfully.
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.objects.instance [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'resources' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFaRzDIjymQDKgKuJGVKrY9YoDm6sxEMtltnJTNSG+jb5Yzbpdqb24ukF2QSYjCu5ZtFM9xpqKafHtv/J2C+itYCqUfzijhnmtqKn1C/3bTReImbLmTyuHqlHmJ8FHo8QQ==',key_name='tempest-TestShelveInstance-472203162',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:26:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=<?>,task_state='shelving',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:26:30Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_vif [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa0610502-2a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-changed-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing instance network info cache due to event network-changed-a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing network info cache for port a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:28:14 user nova-compute[71323]: INFO os_vif [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a')
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:28:14 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6695348-6c91-46ca-8ecd-73b6bf73479c]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:28:14 user nova-compute[71323]: No targets found.
Apr 14 03:28:14 user nova-compute[71323]: iSNS:
Apr 14 03:28:14 user nova-compute[71323]: No targets found.
Apr 14 03:28:14 user nova-compute[71323]: STATIC:
Apr 14 03:28:14 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:28:14 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:28:14 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:28:14 user nova-compute[71323]: FIRMWARE:
Apr 14 03:28:14 user nova-compute[71323]: No targets found.
Apr 14 03:28:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:28:14 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:28:14 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5efac4f3-9e3d-48fc-8ae2-a1063285d88f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbf247a3-6c0d-4424-981d-27935e7a926e]: (4, ('tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:28:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:28:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b007df760>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1153d46c-3c9b-4159-92aa-690cfab81b4f]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e24676d-c15d-43eb-9668-d7fa16bf1aad]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[306547ae-494c-45f9-b3ea-ab0b891e0ac4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfdc1902-e29b-4338-9abe-d6119cf1d08d]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f850967e-9083-4eea-9e00-07e388cfe298]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --logout" returned: 0 in 0.045s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd1b9c13-b3ba-45e6-a260-e6c596e8c700]: (4, ('Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]\nLogout of [sid: 50, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]
Apr 14 03:28:14 user nova-compute[71323]: Logout of [sid: 50, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.
Apr 14 03:28:14 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d6273b0-dfcd-464f-92dc-0b0fa1a33174]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.281s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== disconnect_volume: return (282ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:28:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deleting instance files /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0_del
Apr 14 03:28:14 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deletion of /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0_del complete
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated VIF entry in instance network info cache for port a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.network.neutron [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": null, "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapa0610502-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f756b9bd-7c78-4334-b8bb-4e77633daeb4 req-9fd61480-e117-4e76-9a64-6bc41f49230b service nova] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:14 user nova-compute[71323]: DEBUG nova.compute.manager [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:14 user nova-compute[71323]: WARNING nova.compute.manager [req-5ef7a3cf-f1fa-4111-b32f-ad16024f0d99 req-4cb4a403-78d7-49fe-99a4-d52b417c9b2a service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state active and task_state shelving.
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.152s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:28:15 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8574MB free_disk=26.322376251220703GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=768MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:28:15 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Deleted allocations for instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.225s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.036s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.objects.instance [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'numa_topology' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:28:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.115s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-efcf0a7f-c9bf-428d-a017-4790e845a63a tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" "released" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: held 3.348s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Skipping network cache update for instance because it has been migrated to another host. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9836}}
Apr 14 03:28:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:28:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:28:28 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442893.3997788, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:28:28 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Stopped (Lifecycle Event)
Apr 14 03:28:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-65cf4066-49be-4b14-a733-79586b96f639 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:32 user nova-compute[71323]: INFO nova.compute.manager [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Terminating instance
Apr 14 03:28:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.compute.manager [req-4f2eb046-56e8-4775-bcad-1946f0c3d3af req-3971b59b-072d-4d31-9546-d5b37449ddd5 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-unplugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:28:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Instance destroyed successfully.
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lazy-loading 'resources' on Instance uuid f8042c4c-0ff6-4533-8f59-de5c41ede33f {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:23:38Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1949831724',display_name='tempest-TestMinimumBasicScenario-server-1949831724',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1949831724',id=41,image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbABr/vH+9uc5wQJ3ZeT8Ydil5OovYfdLqGJBQSVbmCHz/2Slj13rAwCGE1mbPPo0/CwXQms3UgRTIS6Sa1TpYWM58n3w3eYIRd+WO/4CHQY+ShdxeZ/KOMF7laMdnfUw==',key_name='tempest-TestMinimumBasicScenario-35983132',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:23:45Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='0bd451b3f6734fe3ac0a8e8a853bb9eb',ramdisk_id='',reservation_id='r-3ckb0aae',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='eed6e9a9-28a5-4022-b718-e7f64bbf7dfd',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-144187586',owner_user_name='tempest-TestMinimumBasicScenario-144187586-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:25:52Z,user_data=None,user_id='12199a43298942898a2e694ee2c5230e',uuid=f8042c4c-0ff6-4533-8f59-de5c41ede33f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converting VIF {"id": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "address": "fa:16:3e:94:e8:44", "network": {"id": "3b75178e-4dcf-4cc1-b910-443fadcadac4", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1386187899-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "0bd451b3f6734fe3ac0a8e8a853bb9eb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8a4c167f-31", "ovs_interfaceid": "8a4c167f-3166-4f90-91c9-1a1d5be3c92d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG os_vif [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8a4c167f-31, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:33 user nova-compute[71323]: INFO os_vif [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:94:e8:44,bridge_name='br-int',has_traffic_filtering=True,id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d,network=Network(3b75178e-4dcf-4cc1-b910-443fadcadac4),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8a4c167f-31')
Apr 14 03:28:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Deleting instance files /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f_del
Apr 14 03:28:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Deletion of /opt/stack/data/nova/instances/f8042c4c-0ff6-4533-8f59-de5c41ede33f_del complete
Apr 14 03:28:33 user nova-compute[71323]: INFO nova.compute.manager [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Took 0.98 seconds to destroy the instance on the hypervisor.
Apr 14 03:28:33 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:28:33 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:34 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Took 0.56 seconds to deallocate network for instance.
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-d4d30d42-cafb-49a7-889f-4891d3b93e22 req-518fa8a8-0292-4132-8cf9-77a61e96a948 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-deleted-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:34 user nova-compute[71323]: INFO nova.compute.manager [req-d4d30d42-cafb-49a7-889f-4891d3b93e22 req-518fa8a8-0292-4132-8cf9-77a61e96a948 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Neutron deleted interface 8a4c167f-3166-4f90-91c9-1a1d5be3c92d; detaching it from the instance and deleting it from the info cache
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.network.neutron [req-d4d30d42-cafb-49a7-889f-4891d3b93e22 req-518fa8a8-0292-4132-8cf9-77a61e96a948 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.compute.manager [req-d4d30d42-cafb-49a7-889f-4891d3b93e22 req-518fa8a8-0292-4132-8cf9-77a61e96a948 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Detach interface failed, port_id=8a4c167f-3166-4f90-91c9-1a1d5be3c92d, reason: Instance f8042c4c-0ff6-4533-8f59-de5c41ede33f could not be found. {{(pid=71323) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10816}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:28:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.148s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:34 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Deleted allocations for instance f8042c4c-0ff6-4533-8f59-de5c41ede33f
Apr 14 03:28:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-a7e9aa8b-0e50-4521-a8fe-0b5fa825593d tempest-TestMinimumBasicScenario-144187586 tempest-TestMinimumBasicScenario-144187586-project-member] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.870s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] Acquiring lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] Lock "f8042c4c-0ff6-4533-8f59-de5c41ede33f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] No waiting events found dispatching network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:35 user nova-compute[71323]: WARNING nova.compute.manager [req-401d1f6a-de3f-42a0-b3e1-5d672c91d950 req-e48f24b8-8a45-495c-a7cf-dc4450c9b0e8 service nova] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Received unexpected event network-vif-plugged-8a4c167f-3166-4f90-91c9-1a1d5be3c92d for instance with vm_state deleted and task_state None.
Apr 14 03:28:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:43 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:28:43 user nova-compute[71323]: INFO nova.compute.claims [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Claim successful on node user
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.217s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:28:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:28:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Creating image(s)
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "/opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "/opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.policy [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c1390cde350341c3ac0275e429146cf8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b689b65111045168cf21c51bf1274f2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk 1073741824" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.195s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking if we can resize image /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Cannot resize image /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'migration_context' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Ensure instance console log exists: /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Successfully created port: a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Successfully updated port: a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquired lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-changed-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Refreshing instance network info cache due to event network-changed-a022509f-61ed-4733-8c46-681c8404e205. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] Acquiring lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:28:45 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating instance_info_cache with network_info: [{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Releasing lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Instance network_info: |[{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] Acquired lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Refreshing network info cache for port a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Start _get_guest_xml network_info=[{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:28:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:28:46 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:28:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-615458774',display_name='tempest-AttachVolumeNegativeTest-server-615458774',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-615458774',id=45,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+zxZZ5GMAcUkiDF/IM7NIX6IijptgrPizAuOU/j3R7zyDZnX1nJ/MDmrrh8943MffhteeJQmerIBtPnXsmcrCsXT4bq4THSz5cImqnpGKxesbrIxKwITwwkD9M1xlRgQ==',key_name='tempest-keypair-593638331',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-tjx5xow7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:28:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=09ed08b8-18ef-4061-a0e9-9c7f18336749,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'pci_devices' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:28:46 user nova-compute[71323]:   <uuid>09ed08b8-18ef-4061-a0e9-9c7f18336749</uuid>
Apr 14 03:28:46 user nova-compute[71323]:   <name>instance-0000002d</name>
Apr 14 03:28:46 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:28:46 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:28:46 user nova-compute[71323]:   <metadata>
Apr 14 03:28:46 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:28:46 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeNegativeTest-server-615458774</nova:name>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:28:46</nova:creationTime>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:28:46 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:28:46 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:owner>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:user uuid="c1390cde350341c3ac0275e429146cf8">tempest-AttachVolumeNegativeTest-376301777-project-member</nova:user>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:project uuid="3b689b65111045168cf21c51bf1274f2">tempest-AttachVolumeNegativeTest-376301777</nova:project>
Apr 14 03:28:46 user nova-compute[71323]:       </nova:owner>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:28:46 user nova-compute[71323]:       <nova:ports>
Apr 14 03:28:46 user nova-compute[71323]:         <nova:port uuid="a022509f-61ed-4733-8c46-681c8404e205">
Apr 14 03:28:46 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Apr 14 03:28:46 user nova-compute[71323]:         </nova:port>
Apr 14 03:28:46 user nova-compute[71323]:       </nova:ports>
Apr 14 03:28:46 user nova-compute[71323]:     </nova:instance>
Apr 14 03:28:46 user nova-compute[71323]:   </metadata>
Apr 14 03:28:46 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:28:46 user nova-compute[71323]:     <system>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="serial">09ed08b8-18ef-4061-a0e9-9c7f18336749</entry>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="uuid">09ed08b8-18ef-4061-a0e9-9c7f18336749</entry>
Apr 14 03:28:46 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:28:46 user nova-compute[71323]:     </system>
Apr 14 03:28:46 user nova-compute[71323]:   </sysinfo>
Apr 14 03:28:46 user nova-compute[71323]:   <os>
Apr 14 03:28:46 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:28:46 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:28:46 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:28:46 user nova-compute[71323]:   </os>
Apr 14 03:28:46 user nova-compute[71323]:   <features>
Apr 14 03:28:46 user nova-compute[71323]:     <acpi/>
Apr 14 03:28:46 user nova-compute[71323]:     <apic/>
Apr 14 03:28:46 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:28:46 user nova-compute[71323]:   </features>
Apr 14 03:28:46 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:28:46 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:28:46 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:28:46 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:28:46 user nova-compute[71323]:   </clock>
Apr 14 03:28:46 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:28:46 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:28:46 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:28:46 user nova-compute[71323]:   </cpu>
Apr 14 03:28:46 user nova-compute[71323]:   <devices>
Apr 14 03:28:46 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:28:46 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:28:46 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk"/>
Apr 14 03:28:46 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:28:46 user nova-compute[71323]:     </disk>
Apr 14 03:28:46 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:28:46 user nova-compute[71323]:       <mac address="fa:16:3e:6c:96:fc"/>
Apr 14 03:28:46 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:28:46 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:28:46 user nova-compute[71323]:       <target dev="tapa022509f-61"/>
Apr 14 03:28:46 user nova-compute[71323]:     </interface>
Apr 14 03:28:46 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:28:46 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/console.log" append="off"/>
Apr 14 03:28:46 user nova-compute[71323]:     </serial>
Apr 14 03:28:46 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:28:46 user nova-compute[71323]:     <video>
Apr 14 03:28:46 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:28:46 user nova-compute[71323]:     </video>
Apr 14 03:28:46 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:28:46 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:28:46 user nova-compute[71323]:     </rng>
Apr 14 03:28:46 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:28:46 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:28:46 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:28:46 user nova-compute[71323]:     </memballoon>
Apr 14 03:28:46 user nova-compute[71323]:   </devices>
Apr 14 03:28:46 user nova-compute[71323]: </domain>
Apr 14 03:28:46 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:28:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-615458774',display_name='tempest-AttachVolumeNegativeTest-server-615458774',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-615458774',id=45,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+zxZZ5GMAcUkiDF/IM7NIX6IijptgrPizAuOU/j3R7zyDZnX1nJ/MDmrrh8943MffhteeJQmerIBtPnXsmcrCsXT4bq4THSz5cImqnpGKxesbrIxKwITwwkD9M1xlRgQ==',key_name='tempest-keypair-593638331',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-tjx5xow7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:28:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=09ed08b8-18ef-4061-a0e9-9c7f18336749,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG os_vif [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa022509f-61, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa022509f-61, col_values=(('external_ids', {'iface-id': 'a022509f-61ed-4733-8c46-681c8404e205', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6c:96:fc', 'vm-uuid': '09ed08b8-18ef-4061-a0e9-9c7f18336749'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:46 user nova-compute[71323]: INFO os_vif [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61')
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:6c:96:fc, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updated VIF entry in instance network info cache for port a022509f-61ed-4733-8c46-681c8404e205. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG nova.network.neutron [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating instance_info_cache with network_info: [{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:28:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cfed2fa9-9f6c-4beb-955d-8177ba18b142 req-06ab7cd3-1641-4173-adf2-34817224e49e service nova] Releasing lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:47 user nova-compute[71323]: DEBUG nova.compute.manager [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] No waiting events found dispatching network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:47 user nova-compute[71323]: WARNING nova.compute.manager [req-507346d5-bc1f-418b-ad0f-6c8238c613bd req-c97bbebd-771f-4eac-94d8-e1fd2328d183 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received unexpected event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 for instance with vm_state building and task_state spawning.
Apr 14 03:28:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:48 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681442913.6681225, f8042c4c-0ff6-4533-8f59-de5c41ede33f => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:28:48 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] VM Stopped (Lifecycle Event)
Apr 14 03:28:48 user nova-compute[71323]: DEBUG nova.compute.manager [None req-44b871a4-3745-4277-9901-b5a24c650a8d None None] [instance: f8042c4c-0ff6-4533-8f59-de5c41ede33f] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442929.6391933, 09ed08b8-18ef-4061-a0e9-9c7f18336749 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] VM Resumed (Lifecycle Event)
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Instance spawned successfully.
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442929.640447, 09ed08b8-18ef-4061-a0e9-9c7f18336749 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] VM Started (Lifecycle Event)
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Took 5.44 seconds to spawn the instance on the hypervisor.
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:49 user nova-compute[71323]: INFO nova.compute.manager [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Took 5.97 seconds to build instance.
Apr 14 03:28:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3a59ecd7-3c28-44fa-9aec-70ab6cad52a1 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.068s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:28:49 user nova-compute[71323]: DEBUG nova.compute.manager [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] No waiting events found dispatching network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:28:49 user nova-compute[71323]: WARNING nova.compute.manager [req-ec677fcd-356a-4cf8-a4c3-634ed164fbf2 req-ea714c53-7e34-450f-aab7-15cbb582db6c service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received unexpected event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 for instance with vm_state active and task_state None.
Apr 14 03:28:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:28:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:29:01 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:11 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:29:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:16 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:17 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:29:17 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8571MB free_disk=26.30480194091797GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:29:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.254s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:29:21 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:22 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating instance_info_cache with network_info: [{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:29:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:29:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:29:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:26 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:26 user nova-compute[71323]: INFO nova.compute.manager [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Unshelving
Apr 14 03:29:27 user nova-compute[71323]: INFO nova.virt.block_device [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Booting with volume 206ed030-e355-423f-a35a-3598ef189cce at /dev/vdc
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.utils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[248dd28a-13d8-4c67-a376-8171e9370109]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1e5f53d-13f2-4544-b5e5-2a222c3d116a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[704a58e6-f288-47bf-84fb-3dc147f06659]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f584baca-88f6-4bf0-a7ed-b6e4a8ee239c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:29:27 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:29:27 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7ad4a33-4f06-41c7-b45a-a0cf60527552]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG os_brick.utils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:29:27 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating existing volume attachment record: 2f84256b-41b3-4369-bf80-046385b17ed3 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:29:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:29:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_requests' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'numa_topology' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:29:31 user nova-compute[71323]: INFO nova.compute.claims [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Claim successful on node user
Apr 14 03:29:31 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:29:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.300s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:31 user nova-compute[71323]: INFO nova.network.neutron [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG nova.compute.manager [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing instance network info cache due to event network-changed-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:29:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:29:33 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Creating image(s)
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'trusted_certs' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG nova.network.neutron [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Refreshing network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.part --force-share --output=json" returned: 0 in 0.190s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG nova.virt.images [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] ea29233a-8154-40d5-89bf-d522c0d1c520 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:29:34 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.part /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updated VIF entry in instance network info cache for port b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-555fd176-2ad5-49ee-b06d-bede33bb51f1 req-45ccf5d4-5c7c-44e9-aa00-15a04d28092c service nova] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.part /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.converted" returned: 0 in 0.629s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49.converted --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.116s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:35 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49,backing_fmt=raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49,backing_fmt=raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk 1073741824" returned: 0 in 0.052s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.193s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49 --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'migration_context' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:36 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Rebasing disk image.
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:36 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d -F raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d -F raw /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk" returned: 0 in 2.010s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Ensure instance console log exists: /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start _get_guest_xml network_info=[{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='3254ae6de7e5445acacb72f9cc5816e9',container_format='bare',created_at=2023-04-14T03:27:35Z,direct_url=<?>,disk_format='qcow2',id=ea29233a-8154-40d5-89bf-d522c0d1c520,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-240447376-shelved',owner='7ed26668c2cb49c38d16ff728d38bef2',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T03:27:38Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '2f84256b-41b3-4369-bf80-046385b17ed3', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FkuMDwZI', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attached', 'instance': '16e9b236-ce8f-4231-af55-e694bc43dde4', 'attached_at': '', 'detached_at': '', 'volume_id': '206ed030-e355-423f-a35a-3598ef189cce', 'serial': '206ed030-e355-423f-a35a-3598ef189cce'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vdc', 'delete_on_termination': False, 'boot_index': None, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:29:38 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:29:38 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='3254ae6de7e5445acacb72f9cc5816e9',container_format='bare',created_at=2023-04-14T03:27:35Z,direct_url=<?>,disk_format='qcow2',id=ea29233a-8154-40d5-89bf-d522c0d1c520,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-240447376-shelved',owner='7ed26668c2cb49c38d16ff728d38bef2',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T03:27:38Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'vcpu_model' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FkuMDwZI', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:29:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:29:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:29:38 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7c94e86-fb7f-4be7-be07-7abd4962de23]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:29:38 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[73dfb544-6def-4ac9-aec4-01b34f1a117a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce] added
Apr 14 03:29:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[73ecef54-adf5-4b99-ae24-b06ca68101ce]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6299823f-94b5-4017-9ea9-ffb0e5893d59]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FkuMDwZI {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v FkuMDwZI" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e3c334a-6b48-4083-a295-23ede09479c9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FkuMDwZI'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fcd8e18-5d11-4912-bf9e-c691816dad94]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[36b9736e-32ba-4bf9-94c5-7de01be59e78]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:29:38 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Apr 14 03:29:38 user nova-compute[71323]:  {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:29:38 user nova-compute[71323]: WARNING os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Apr 14 03:29:38 user nova-compute[71323]: 
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --login" returned: 0 in 0.045s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d07526f-f2f8-4c5d-ab45-a859a249166c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260]
Apr 14 03:29:38 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260] successful.
Apr 14 03:29:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f3f46e9-0da8-4dc2-b89f-bf40e49a4d6f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9082c0a1-666d-4d3c-9a4d-c548d2544907]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:29:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:29:38 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.269s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] HCTL ('33', '-', '-', 0) found on session 52 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd9bd61b-72f8-4b29-bbbd-80e46fd5af9d]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:29:38 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Searching for a device in session 52 and hctl ['33', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Searching for a device in session 52 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FkuMDwZI', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.298s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] <== connect_volume: return (1299ms) {'type': 'block', 'scsi_wwn': '26266363739303934', 'path': '/dev/sdb'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26266363739303934', 'path': '/dev/sdb'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='ea29233a-8154-40d5-89bf-d522c0d1c520',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1931660222',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:25:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:27:38.542369',shelved_host='user',shelved_image_id='ea29233a-8154-40d5-89bf-d522c0d1c520'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:29:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.objects.instance [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_devices' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:29:39 user nova-compute[71323]:   <uuid>16e9b236-ce8f-4231-af55-e694bc43dde4</uuid>
Apr 14 03:29:39 user nova-compute[71323]:   <name>instance-0000002b</name>
Apr 14 03:29:39 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:29:39 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:29:39 user nova-compute[71323]:   <metadata>
Apr 14 03:29:39 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:29:39 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-240447376</nova:name>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:29:38</nova:creationTime>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:29:39 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:29:39 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:owner>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:user uuid="f88237b825b24effad17900bbf645ab8">tempest-AttachVolumeShelveTestJSON-1546459049-project-member</nova:user>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:project uuid="7ed26668c2cb49c38d16ff728d38bef2">tempest-AttachVolumeShelveTestJSON-1546459049</nova:project>
Apr 14 03:29:39 user nova-compute[71323]:       </nova:owner>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:root type="image" uuid="ea29233a-8154-40d5-89bf-d522c0d1c520"/>
Apr 14 03:29:39 user nova-compute[71323]:       <nova:ports>
Apr 14 03:29:39 user nova-compute[71323]:         <nova:port uuid="b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4">
Apr 14 03:29:39 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Apr 14 03:29:39 user nova-compute[71323]:         </nova:port>
Apr 14 03:29:39 user nova-compute[71323]:       </nova:ports>
Apr 14 03:29:39 user nova-compute[71323]:     </nova:instance>
Apr 14 03:29:39 user nova-compute[71323]:   </metadata>
Apr 14 03:29:39 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:29:39 user nova-compute[71323]:     <system>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="serial">16e9b236-ce8f-4231-af55-e694bc43dde4</entry>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="uuid">16e9b236-ce8f-4231-af55-e694bc43dde4</entry>
Apr 14 03:29:39 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:29:39 user nova-compute[71323]:     </system>
Apr 14 03:29:39 user nova-compute[71323]:   </sysinfo>
Apr 14 03:29:39 user nova-compute[71323]:   <os>
Apr 14 03:29:39 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:29:39 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:29:39 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:29:39 user nova-compute[71323]:   </os>
Apr 14 03:29:39 user nova-compute[71323]:   <features>
Apr 14 03:29:39 user nova-compute[71323]:     <acpi/>
Apr 14 03:29:39 user nova-compute[71323]:     <apic/>
Apr 14 03:29:39 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:29:39 user nova-compute[71323]:   </features>
Apr 14 03:29:39 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:29:39 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:29:39 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:29:39 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:29:39 user nova-compute[71323]:   </clock>
Apr 14 03:29:39 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:29:39 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:29:39 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:29:39 user nova-compute[71323]:   </cpu>
Apr 14 03:29:39 user nova-compute[71323]:   <devices>
Apr 14 03:29:39 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:29:39 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:29:39 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk"/>
Apr 14 03:29:39 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:29:39 user nova-compute[71323]:     </disk>
Apr 14 03:29:39 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:29:39 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:29:39 user nova-compute[71323]:       <source dev="/dev/sdb"/>
Apr 14 03:29:39 user nova-compute[71323]:       <target dev="vdc" bus="virtio"/>
Apr 14 03:29:39 user nova-compute[71323]:       <serial>206ed030-e355-423f-a35a-3598ef189cce</serial>
Apr 14 03:29:39 user nova-compute[71323]:     </disk>
Apr 14 03:29:39 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:29:39 user nova-compute[71323]:       <mac address="fa:16:3e:11:d8:7b"/>
Apr 14 03:29:39 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:29:39 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:29:39 user nova-compute[71323]:       <target dev="tapb47d2f30-8e"/>
Apr 14 03:29:39 user nova-compute[71323]:     </interface>
Apr 14 03:29:39 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:29:39 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/console.log" append="off"/>
Apr 14 03:29:39 user nova-compute[71323]:     </serial>
Apr 14 03:29:39 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:29:39 user nova-compute[71323]:     <video>
Apr 14 03:29:39 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:29:39 user nova-compute[71323]:     </video>
Apr 14 03:29:39 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:29:39 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:29:39 user nova-compute[71323]:     </rng>
Apr 14 03:29:39 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:29:39 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:29:39 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:29:39 user nova-compute[71323]:     </memballoon>
Apr 14 03:29:39 user nova-compute[71323]:   </devices>
Apr 14 03:29:39 user nova-compute[71323]: </domain>
Apr 14 03:29:39 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='ea29233a-8154-40d5-89bf-d522c0d1c520',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-1931660222',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:25:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:27:38.542369',shelved_host='user',shelved_image_id='ea29233a-8154-40d5-89bf-d522c0d1c520'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:29:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG os_vif [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb47d2f30-8e, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapb47d2f30-8e, col_values=(('external_ids', {'iface-id': 'b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:11:d8:7b', 'vm-uuid': '16e9b236-ce8f-4231-af55-e694bc43dde4'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:39 user nova-compute[71323]: INFO os_vif [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e')
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No BDM found with device name vdc, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:29:39 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No VIF found with MAC fa:16:3e:11:d8:7b, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:29:41 user nova-compute[71323]: WARNING nova.compute.manager [req-cec9eed9-a8fc-453e-ac74-6298fa062596 req-c44f8dc0-0a3f-4ce6-afbd-f11ffdd0912b service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:41 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442983.4615946, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:29:43 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Resumed (Lifecycle Event)
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:29:43 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance spawned successfully.
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:29:43 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681442983.4646814, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:29:43 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Started (Lifecycle Event)
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:29:43 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:29:43 user nova-compute[71323]: WARNING nova.compute.manager [req-d1e62d02-7705-4a9b-94c4-41160bfa0e81 req-30ad7c50-56ab-4b1c-8719-1f4b1033be82 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 03:29:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:29:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-82abefc7-c9d8-4565-85f6-f54cce5db684 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 16.939s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:29:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:29:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:29:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:29:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:04 user nova-compute[71323]: INFO nova.compute.manager [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Unshelving
Apr 14 03:30:04 user nova-compute[71323]: INFO nova.virt.block_device [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Booting with volume 3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 at /dev/vda
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.utils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.023s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[41629a57-a79b-4273-8c7d-3fc562604d44]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd09d011-e624-45f3-8299-8b9bde0cadff]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[efb682ea-ae96-495d-b01a-9d7ff515b209]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[40c5c9fc-6729-4eef-8e98-418f82dd13f6]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:30:04 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:30:04 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5da7749-7a03-4acd-ac72-733d55e028ea]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG os_brick.utils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating existing volume attachment record: 007d4e50-5e3e-4810-9926-671483ce4061 {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:08 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:08 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'pci_requests' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:08 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'numa_topology' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:08 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:30:08 user nova-compute[71323]: INFO nova.compute.claims [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Claim successful on node user
Apr 14 03:30:09 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:30:09 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:30:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.251s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:09 user nova-compute[71323]: INFO nova.network.neutron [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating port a0610502-2adf-475c-822c-d96d457872a8 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 03:30:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-changed-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.compute.manager [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing instance network info cache due to event network-changed-a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.network.neutron [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:30:10 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Creating image(s)
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Did not create local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4834}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Ensure instance console log exists: /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start _get_guest_xml network_info=[{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'device_type': 'disk', 'attachment_id': '007d4e50-5e3e-4810-9926-671483ce4061', 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ce889d46-fac9-4f77-9d66-6e7753d8a0e0', 'attached_at': '', 'detached_at': '', 'volume_id': '3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'serial': '3a8e1f1a-36b6-47b7-95f2-d31cd1b55808'}, 'disk_bus': 'virtio', 'guest_format': None, 'mount_device': '/dev/vda', 'delete_on_termination': True, 'boot_index': 0, 'volume_type': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.network.neutron [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing network info cache for port a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:30:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:30:10 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'vcpu_model' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:30:10 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:30:10 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:30:10 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[933a7b2c-30e6-4470-99cc-423f01bc38e1]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:30:10 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.031s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c7795a9-6d2a-4639-be39-d862c41502da]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808] added
Apr 14 03:30:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f2f8fb9-4de8-492b-8cd0-db75d971b5a4]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8996ac83-d307-463e-b1f0-9ee2979de495]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TcJSZhIU {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v TcJSZhIU" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0371229-b2a6-4490-874d-cef25c70c33e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'TcJSZhIU'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[84560b38-f6ce-438c-9a3e-7b2551dbf5c1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[83b9d6b7-7e43-4c3a-b8c3-ebe7faa531a3]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:10 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:30:10 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --login" returned: 0 in 0.052s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7113427-44c6-412a-8ca5-d20aab0c05db]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]
Apr 14 03:30:11 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.
Apr 14 03:30:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e1a9f06-6446-41bb-a9b7-0b6186f989ee]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[cfbeac35-2937-42d2-990b-3cca9034305c]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:11 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:11 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:11 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.298s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] HCTL ('34', '-', '-', 0) found on session 53 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[6dee44da-5e2a-415c-9505-b722f8bd68b9]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Searching for a device in session 53 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated VIF entry in instance network info cache for port a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG nova.network.neutron [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-34a36785-944b-4eb6-b8e6-8e8f913265eb req-0d9dba25-dd87-43f6-a48e-0bfd12d0a077 service nova] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:11 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.333s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== connect_volume: return (1334ms) {'type': 'block', 'scsi_wwn': '23264633231623239', 'path': '/dev/sdc'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23264633231623239', 'path': '/dev/sdc'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-472203162',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:26:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:30:04Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.objects.instance [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'pci_devices' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:30:12 user nova-compute[71323]:   <uuid>ce889d46-fac9-4f77-9d66-6e7753d8a0e0</uuid>
Apr 14 03:30:12 user nova-compute[71323]:   <name>instance-0000002c</name>
Apr 14 03:30:12 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:30:12 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:30:12 user nova-compute[71323]:   <metadata>
Apr 14 03:30:12 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:30:12 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:30:12 user nova-compute[71323]:       <nova:name>tempest-TestShelveInstance-server-1348709575</nova:name>
Apr 14 03:30:12 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:30:10</nova:creationTime>
Apr 14 03:30:12 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:30:12 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:30:12 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:30:12 user nova-compute[71323]:       <nova:owner>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:user uuid="61f477304a044148ade1c6acffd30712">tempest-TestShelveInstance-1966208901-project-member</nova:user>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:project uuid="1e0979cf1834435bb0007899da52e863">tempest-TestShelveInstance-1966208901</nova:project>
Apr 14 03:30:12 user nova-compute[71323]:       </nova:owner>
Apr 14 03:30:12 user nova-compute[71323]:       <nova:ports>
Apr 14 03:30:12 user nova-compute[71323]:         <nova:port uuid="a0610502-2adf-475c-822c-d96d457872a8">
Apr 14 03:30:12 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Apr 14 03:30:12 user nova-compute[71323]:         </nova:port>
Apr 14 03:30:12 user nova-compute[71323]:       </nova:ports>
Apr 14 03:30:12 user nova-compute[71323]:     </nova:instance>
Apr 14 03:30:12 user nova-compute[71323]:   </metadata>
Apr 14 03:30:12 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:30:12 user nova-compute[71323]:     <system>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="serial">ce889d46-fac9-4f77-9d66-6e7753d8a0e0</entry>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="uuid">ce889d46-fac9-4f77-9d66-6e7753d8a0e0</entry>
Apr 14 03:30:12 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:30:12 user nova-compute[71323]:     </system>
Apr 14 03:30:12 user nova-compute[71323]:   </sysinfo>
Apr 14 03:30:12 user nova-compute[71323]:   <os>
Apr 14 03:30:12 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:30:12 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:30:12 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:30:12 user nova-compute[71323]:   </os>
Apr 14 03:30:12 user nova-compute[71323]:   <features>
Apr 14 03:30:12 user nova-compute[71323]:     <acpi/>
Apr 14 03:30:12 user nova-compute[71323]:     <apic/>
Apr 14 03:30:12 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:30:12 user nova-compute[71323]:   </features>
Apr 14 03:30:12 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:30:12 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:30:12 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:30:12 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:30:12 user nova-compute[71323]:   </clock>
Apr 14 03:30:12 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:30:12 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:30:12 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:30:12 user nova-compute[71323]:   </cpu>
Apr 14 03:30:12 user nova-compute[71323]:   <devices>
Apr 14 03:30:12 user nova-compute[71323]:     <disk type="block" device="disk">
Apr 14 03:30:12 user nova-compute[71323]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:30:12 user nova-compute[71323]:       <source dev="/dev/sdc"/>
Apr 14 03:30:12 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:30:12 user nova-compute[71323]:       <serial>3a8e1f1a-36b6-47b7-95f2-d31cd1b55808</serial>
Apr 14 03:30:12 user nova-compute[71323]:     </disk>
Apr 14 03:30:12 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:30:12 user nova-compute[71323]:       <mac address="fa:16:3e:02:35:6f"/>
Apr 14 03:30:12 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:30:12 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:30:12 user nova-compute[71323]:       <target dev="tapa0610502-2a"/>
Apr 14 03:30:12 user nova-compute[71323]:     </interface>
Apr 14 03:30:12 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:30:12 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0/console.log" append="off"/>
Apr 14 03:30:12 user nova-compute[71323]:     </serial>
Apr 14 03:30:12 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:30:12 user nova-compute[71323]:     <video>
Apr 14 03:30:12 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:30:12 user nova-compute[71323]:     </video>
Apr 14 03:30:12 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:30:12 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:30:12 user nova-compute[71323]:     </rng>
Apr 14 03:30:12 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:30:12 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:30:12 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:30:12 user nova-compute[71323]:     </memballoon>
Apr 14 03:30:12 user nova-compute[71323]:   </devices>
Apr 14 03:30:12 user nova-compute[71323]: </domain>
Apr 14 03:30:12 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-472203162',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:26:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:30:04Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG os_vif [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa0610502-2a, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapa0610502-2a, col_values=(('external_ids', {'iface-id': 'a0610502-2adf-475c-822c-d96d457872a8', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:02:35:6f', 'vm-uuid': 'ce889d46-fac9-4f77-9d66-6e7753d8a0e0'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:12 user nova-compute[71323]: INFO os_vif [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a')
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:30:12 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] No VIF found with MAC fa:16:3e:02:35:6f, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:13 user nova-compute[71323]: DEBUG nova.compute.manager [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:30:13 user nova-compute[71323]: WARNING nova.compute.manager [req-18a58f14-0f32-468c-a468-4d5463b23d7f req-379651df-1b83-4125-a42e-df100d899a6b service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 03:30:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443015.549719, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:30:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Resumed (Lifecycle Event)
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:30:15 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance spawned successfully.
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:30:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443015.5512052, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:30:15 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Started (Lifecycle Event)
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-e491ff9a-a99a-4cc6-a075-e129c39966e3 tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 11.393s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:15 user nova-compute[71323]: DEBUG nova.compute.manager [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:30:15 user nova-compute[71323]: WARNING nova.compute.manager [req-b59d3c8b-d071-4bc8-b04f-793f985d47cd req-4031a682-e897-4b90-933d-8550323790bb service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state active and task_state None.
Apr 14 03:30:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:17 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:30:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.147s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.141s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdb (vdc) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:30:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8496MB free_disk=26.30675506591797GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 16e9b236-ce8f-4231-af55-e694bc43dde4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:30:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:30:20 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:30:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:30:20 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:30:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.255s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:30:23 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [{"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:30:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-16e9b236-ce8f-4231-af55-e694bc43dde4" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:30:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:30:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:30:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-changed-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Refreshing instance network info cache due to event network-changed-a022509f-61ed-4733-8c46-681c8404e205. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] Acquiring lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] Acquired lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Refreshing network info cache for port a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updated VIF entry in instance network info cache for port a022509f-61ed-4733-8c46-681c8404e205. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG nova.network.neutron [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating instance_info_cache with network_info: [{"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:30:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5af207fc-e670-4b02-99ca-f76cd9323052 req-a964c39d-7e65-4f5e-8185-55f6ad90a5c6 service nova] Releasing lock "refresh_cache-09ed08b8-18ef-4061-a0e9-9c7f18336749" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG nova.objects.instance [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:37 user nova-compute[71323]: INFO nova.compute.manager [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Attaching volume 52935d67-1254-4ae3-810b-7b1ffa098955 to /dev/vdb
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.utils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c3cd2cb-7316-45c1-b5be-7f38e5433980]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:e1198c5ebb57\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[09ebcd09-3968-4ecc-964a-ee2593250c79]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b84a342-5997-49c2-af9c-8c44b73ac848]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[05a047e3-c8f9-41f8-a8af-e07064511917]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Running cmd (subprocess): nvme version {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] 'nvme version' failed. Not Retrying. {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.nvmeof [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] nvme not present on system {{(pid=71323) nvme_present /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/nvmeof.py:757}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:473}}
Apr 14 03:30:38 user nova-compute[71323]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Apr 14 03:30:38 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe4842d1-f759-47de-b483-e26b5d98cb46]: (4, '') {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=71323) find_dsc /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:98}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:76}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.lightos [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] LIGHTOS: no hostnqn found. {{(pid=71323) get_connector_properties /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/lightos.py:84}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG os_brick.utils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:e1198c5ebb57', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:30:38 user nova-compute[71323]: DEBUG nova.virt.block_device [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating existing volume attachment record: d1cbd8ee-9826-4935-9562-4f4d7400ba3b {{(pid=71323) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Apr 14 03:30:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Calling os-brick to attach iSCSI Volume {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AENZapny', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:30:42 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:30:42 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:30:42 user nova-compute[71323]: INFO os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfcdea59-befe-45a7-8d2c-f6b755eb04ee]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Apr 14 03:30:42 user nova-compute[71323]:  {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --interface default --op new {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc63fa0d-159a-4621-9148-f72599e72696]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955] added\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955] added
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ffe6018-1f76-423b-a8b0-3f369ec9d289]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8dc668c-293d-408d-9cfc-053773c847d9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AENZapny {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AENZapny" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb9b4116-fbfc-4506-ba4f-9a2a719dd315]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'AENZapny'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2e4f908-ca09-4af4-9672-7eb8da91734e]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbae23f4-9ecd-4a6a-a8d3-71d9b952dd37]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --login {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[db0bbbfe-234e-48ec-adec-0dc14b5a0667]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260]
Apr 14 03:30:42 user nova-compute[71323]: Login to [iface: default, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260] successful.
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ba729ff-d625-4c0a-94d5-ed947769d6a1]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18850b2d-f92f-4002-92e8-4f80b188ed87]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)
Apr 14 03:30:42 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.275s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to 172.16.0.220:3260 {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:633}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] HCTL ('35', '-', '-', 0) found on session 54 with lun 0 {{(pid=71323) get_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:694}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=71323) scan_iscsi /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:719}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[81df67a3-7dde-4517-a532-415570f8f379]: (4, ('- - 0', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:42 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 54 and hctl ['35', '*', '*', 0] yield: None {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Searching for a device in session 54 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=71323) device_name_by_hctl /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:713}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AENZapny', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=71323) _connect_vol /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:661}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.305s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== connect_volume: return (1307ms) {'type': 'block', 'scsi_wwn': '26262356462313330', 'path': '/dev/sdd'} {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26262356462313330', 'path': '/dev/sdd'} {{(pid=71323) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG nova.objects.instance [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] attach device xml: <disk type="block" device="disk">
Apr 14 03:30:43 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:30:43 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:30:43 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:30:43 user nova-compute[71323]:   <serial>52935d67-1254-4ae3-810b-7b1ffa098955</serial>
Apr 14 03:30:43 user nova-compute[71323]: </disk>
Apr 14 03:30:43 user nova-compute[71323]:  {{(pid=71323) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No BDM found with device name vdb, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] No VIF found with MAC fa:16:3e:6c:96:fc, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-dd443342-9a5a-4bb4-b79c-d7014439b281 tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 6.306s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:30:44 user nova-compute[71323]: INFO nova.compute.manager [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Detaching volume 52935d67-1254-4ae3-810b-7b1ffa098955
Apr 14 03:30:44 user nova-compute[71323]: INFO nova.virt.block_device [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Attempting to driver detach volume 52935d67-1254-4ae3-810b-7b1ffa098955 from mountpoint /dev/vdb
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Attempting to detach device vdb from instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:30:44 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:30:44 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:30:44 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:30:44 user nova-compute[71323]:   <serial>52935d67-1254-4ae3-810b-7b1ffa098955</serial>
Apr 14 03:30:44 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:30:44 user nova-compute[71323]: </disk>
Apr 14 03:30:44 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:30:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 from the persistent domain config.
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:30:44 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:30:44 user nova-compute[71323]:   <source dev="/dev/sdd"/>
Apr 14 03:30:44 user nova-compute[71323]:   <target dev="vdb" bus="virtio"/>
Apr 14 03:30:44 user nova-compute[71323]:   <serial>52935d67-1254-4ae3-810b-7b1ffa098955</serial>
Apr 14 03:30:44 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Apr 14 03:30:44 user nova-compute[71323]: </disk>
Apr 14 03:30:44 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:30:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681443045.201732, 09ed08b8-18ef-4061-a0e9-9c7f18336749 => virtio-disk1> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:30:45 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully detached device vdb from instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 from the live domain config.
Apr 14 03:30:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AENZapny', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:30:45 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8960e6a9-8019-4e08-ad38-f596a9ead240]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:30:45 user nova-compute[71323]: No targets found.
Apr 14 03:30:45 user nova-compute[71323]: iSNS:
Apr 14 03:30:45 user nova-compute[71323]: No targets found.
Apr 14 03:30:45 user nova-compute[71323]: STATIC:
Apr 14 03:30:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:30:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:30:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:30:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955
Apr 14 03:30:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:30:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:30:45 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce
Apr 14 03:30:45 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:30:45 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:30:45 user nova-compute[71323]: FIRMWARE:
Apr 14 03:30:45 user nova-compute[71323]: No targets found.
Apr 14 03:30:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:30:45 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:30:45 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[563f4773-78b9-498a-a6a5-7512e9acef31]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d7fed01-f3b4-4fd3-8719-b8008c2a9516]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:45 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:45 user nova-compute[71323]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)
Apr 14 03:30:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:30:45 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:30:45 user nova-compute[71323]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 (non-flash)
Apr 14 03:30:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1ccfe290>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955'): ({'sdd'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Removing single pathed devices sdd {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[da2da91f-b09b-4352-8649-361eb77b47fa]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[87920b03-7581-4872-b2ef-6f34abd248f4]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Flushing IO for device /dev/sdd {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[4842a405-5c2e-458a-aa4d-b4df28ec2a79]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.040s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f72644a-314d-4cd9-99e0-866c3e562c9e]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] SCSI volumes sdd have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd38a471-a8f5-4e96-beb1-6ca64cef684d]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f9f909e-193b-4f01-8992-dbc9263403e2]: (4, ('Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260]\nLogout of [sid: 54, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260]
Apr 14 03:30:45 user nova-compute[71323]: Logout of [sid: 54, target: iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955, portal: 172.16.0.220,3260] successful.
Apr 14 03:30:45 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:52935d67-1254-4ae3-810b-7b1ffa098955 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cb6612a-8422-44c7-8af9-7a317a6c1631]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.297s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] <== disconnect_volume: return (298ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:30:45 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:30:46 user nova-compute[71323]: DEBUG nova.objects.instance [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'flavor' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:30:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-54d6d9e9-cd19-4390-9707-985a9c5dac9b tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.621s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:30:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:30:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:30:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:11 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:17 user nova-compute[71323]: INFO nova.compute.manager [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Detaching volume 206ed030-e355-423f-a35a-3598ef189cce
Apr 14 03:31:17 user nova-compute[71323]: INFO nova.virt.block_device [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Attempting to driver detach volume 206ed030-e355-423f-a35a-3598ef189cce from mountpoint /dev/vdc
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Attempting to detach device vdc from instance 16e9b236-ce8f-4231-af55-e694bc43dde4 from the persistent domain config. {{(pid=71323) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2458}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:31:17 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:31:17 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:31:17 user nova-compute[71323]:   <target dev="vdc" bus="virtio"/>
Apr 14 03:31:17 user nova-compute[71323]:   <serial>206ed030-e355-423f-a35a-3598ef189cce</serial>
Apr 14 03:31:17 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:31:17 user nova-compute[71323]: </disk>
Apr 14 03:31:17 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:31:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully detached device vdc from instance 16e9b236-ce8f-4231-af55-e694bc43dde4 from the persistent domain config.
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 16e9b236-ce8f-4231-af55-e694bc43dde4 from the live domain config. {{(pid=71323) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2494}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.guest [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] detach device xml: <disk type="block" device="disk">
Apr 14 03:31:17 user nova-compute[71323]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Apr 14 03:31:17 user nova-compute[71323]:   <source dev="/dev/sdb"/>
Apr 14 03:31:17 user nova-compute[71323]:   <target dev="vdc" bus="virtio"/>
Apr 14 03:31:17 user nova-compute[71323]:   <serial>206ed030-e355-423f-a35a-3598ef189cce</serial>
Apr 14 03:31:17 user nova-compute[71323]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Apr 14 03:31:17 user nova-compute[71323]: </disk>
Apr 14 03:31:17 user nova-compute[71323]:  {{(pid=71323) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Received event <DeviceRemovedEvent: 1681443077.668436, 16e9b236-ce8f-4231-af55-e694bc43dde4 => virtio-disk2> from libvirt while the driver is waiting for it; dispatched. {{(pid=71323) emit_event /opt/stack/nova/nova/virt/libvirt/driver.py:2341}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2570}}
Apr 14 03:31:17 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully detached device vdc from instance 16e9b236-ce8f-4231-af55-e694bc43dde4 from the live domain config.
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'FkuMDwZI', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:31:17 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a09d8aa9-44ee-44d4-920b-206b3654bee1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:31:17 user nova-compute[71323]: No targets found.
Apr 14 03:31:17 user nova-compute[71323]: iSNS:
Apr 14 03:31:17 user nova-compute[71323]: No targets found.
Apr 14 03:31:17 user nova-compute[71323]: STATIC:
Apr 14 03:31:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:31:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:31:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:31:17 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce
Apr 14 03:31:17 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:31:17 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:31:17 user nova-compute[71323]: FIRMWARE:
Apr 14 03:31:17 user nova-compute[71323]: No targets found.
Apr 14 03:31:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:31:17 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:31:17 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[86a2436f-1e57-4f11-a2fa-761690c63b6d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e524886-e71d-48d7-8140-0af10d39be36]: (4, ('tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:31:17 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:31:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsi session list stdout=tcp: [52] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce (non-flash)
Apr 14 03:31:17 user nova-compute[71323]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:31:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2ac1f42dd0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce'): ({'sdb'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Removing single pathed devices sdb {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c4f51f9-0144-4872-b3bd-59f4e524ba12]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[46beadcd-c42c-4bea-acd7-7e3d94e9a423]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flushing IO for device /dev/sdb {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[15add20c-ffff-4cea-8aa7-c7341570ff6f]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.050s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[10c844cb-73af-40ed-b6b9-bd3364a7ff43]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] SCSI volumes sdb have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[a689eb03-6e20-439c-bed7-769f45277ca6]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[92a5c7e1-f713-4e0b-9175-a84756cb86f0]: (4, ('Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260]\nLogout of [sid: 52, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 52, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260]
Apr 14 03:31:17 user nova-compute[71323]: Logout of [sid: 52, target: iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce, portal: 172.16.0.220,3260] successful.
Apr 14 03:31:17 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:206ed030-e355-423f-a35a-3598ef189cce -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[d78bde80-2494-458a-b2a8-e3aaf33f64f9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.288s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] <== disconnect_volume: return (290ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:31:17 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG nova.objects.instance [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'flavor' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-ef0b10a4-005e-4638-bb4d-733bf3741232 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 1.614s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:31:18 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.145s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749/disk --force-share --output=json" returned: 0 in 0.157s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4/disk --force-share --output=json" returned: 0 in 0.135s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=71323) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11189}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:31:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8483MB free_disk=26.30545425415039GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 09ed08b8-18ef-4061-a0e9-9c7f18336749 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 16e9b236-ce8f-4231-af55-e694bc43dde4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=896MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:31:19 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:31:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.366s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:31:23 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:31:24 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:31:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:31:24 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:31:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:31:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:35 user nova-compute[71323]: INFO nova.compute.manager [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Terminating instance
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-unplugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] No waiting events found dispatching network-vif-unplugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.compute.manager [req-2caf1dfb-4c87-4454-a994-4834d4b47def req-9b19ae03-5463-4fee-9d4f-83e60bef3f66 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-unplugged-a022509f-61ed-4733-8c46-681c8404e205 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Instance destroyed successfully.
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lazy-loading 'resources' on Instance uuid 09ed08b8-18ef-4061-a0e9-9c7f18336749 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:28:43Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-615458774',display_name='tempest-AttachVolumeNegativeTest-server-615458774',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-615458774',id=45,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM+zxZZ5GMAcUkiDF/IM7NIX6IijptgrPizAuOU/j3R7zyDZnX1nJ/MDmrrh8943MffhteeJQmerIBtPnXsmcrCsXT4bq4THSz5cImqnpGKxesbrIxKwITwwkD9M1xlRgQ==',key_name='tempest-keypair-593638331',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:28:49Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='3b689b65111045168cf21c51bf1274f2',ramdisk_id='',reservation_id='r-tjx5xow7',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-376301777',owner_user_name='tempest-AttachVolumeNegativeTest-376301777-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:28:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='c1390cde350341c3ac0275e429146cf8',uuid=09ed08b8-18ef-4061-a0e9-9c7f18336749,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converting VIF {"id": "a022509f-61ed-4733-8c46-681c8404e205", "address": "fa:16:3e:6c:96:fc", "network": {"id": "c6419e57-87af-44c1-a594-3a6a40f976b8", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-2085393478-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.159", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "3b689b65111045168cf21c51bf1274f2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa022509f-61", "ovs_interfaceid": "a022509f-61ed-4733-8c46-681c8404e205", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG os_vif [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa022509f-61, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:31:35 user nova-compute[71323]: INFO os_vif [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6c:96:fc,bridge_name='br-int',has_traffic_filtering=True,id=a022509f-61ed-4733-8c46-681c8404e205,network=Network(c6419e57-87af-44c1-a594-3a6a40f976b8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa022509f-61')
Apr 14 03:31:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Deleting instance files /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749_del
Apr 14 03:31:35 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Deletion of /opt/stack/data/nova/instances/09ed08b8-18ef-4061-a0e9-9c7f18336749_del complete
Apr 14 03:31:35 user nova-compute[71323]: INFO nova.compute.manager [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Took 0.82 seconds to destroy the instance on the hypervisor.
Apr 14 03:31:35 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:31:35 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:31:36 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:31:36 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Took 0.68 seconds to deallocate network for instance.
Apr 14 03:31:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:36 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:31:36 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:31:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.153s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:36 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Deleted allocations for instance 09ed08b8-18ef-4061-a0e9-9c7f18336749
Apr 14 03:31:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-1fbcc52d-54ca-4a48-ada6-48106feb1cef tempest-AttachVolumeNegativeTest-376301777 tempest-AttachVolumeNegativeTest-376301777-project-member] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.827s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] Acquiring lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:37 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] Lock "09ed08b8-18ef-4061-a0e9-9c7f18336749-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] No waiting events found dispatching network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:31:37 user nova-compute[71323]: WARNING nova.compute.manager [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received unexpected event network-vif-plugged-a022509f-61ed-4733-8c46-681c8404e205 for instance with vm_state deleted and task_state None.
Apr 14 03:31:37 user nova-compute[71323]: DEBUG nova.compute.manager [req-f1b6acce-9f53-4d61-a717-17c66b07c7f7 req-0ad78997-da36-4fed-a7a1-73e8d6724dc4 service nova] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Received event network-vif-deleted-a022509f-61ed-4733-8c46-681c8404e205 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:50 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443095.7512217, 09ed08b8-18ef-4061-a0e9-9c7f18336749 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:31:50 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] VM Stopped (Lifecycle Event)
Apr 14 03:31:50 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d60e3f8d-39d6-4913-9b5c-6f1efaf3f7e9 None None] [instance: 09ed08b8-18ef-4061-a0e9-9c7f18336749] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:31:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-changed-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG nova.compute.manager [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing instance network info cache due to event network-changed-a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] Acquiring lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] Acquired lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG nova.network.neutron [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Refreshing network info cache for port a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: INFO nova.compute.manager [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Terminating instance
Apr 14 03:31:54 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.compute.manager [req-1a030c30-e16b-42d2-abb3-4f8b0bb99a76 req-eb1242ac-b520-42a2-8cb8-a0d15efdd0aa service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-unplugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:31:55 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Instance destroyed successfully.
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lazy-loading 'resources' on Instance uuid ce889d46-fac9-4f77-9d66-6e7753d8a0e0 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:26:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1348709575',display_name='tempest-TestShelveInstance-server-1348709575',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1348709575',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFaRzDIjymQDKgKuJGVKrY9YoDm6sxEMtltnJTNSG+jb5Yzbpdqb24ukF2QSYjCu5ZtFM9xpqKafHtv/J2C+itYCqUfzijhnmtqKn1C/3bTReImbLmTyuHqlHmJ8FHo8QQ==',key_name='tempest-TestShelveInstance-472203162',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:30:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='1e0979cf1834435bb0007899da52e863',ramdisk_id='',reservation_id='r-0k19vp7a',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1966208901',owner_user_name='tempest-TestShelveInstance-1966208901-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:30:16Z,user_data=None,user_id='61f477304a044148ade1c6acffd30712',uuid=ce889d46-fac9-4f77-9d66-6e7753d8a0e0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converting VIF {"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.235", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_vif [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa0610502-2a, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:31:55 user nova-compute[71323]: INFO os_vif [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:35:6f,bridge_name='br-int',has_traffic_filtering=True,id=a0610502-2adf-475c-822c-d96d457872a8,network=Network(f0e6e99b-d215-4622-9a1c-028eca82f6a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa0610502-2a')
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] calling os-brick to detach iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f2b00280790>, {'target_iqn': 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'TcJSZhIU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc': {}}" {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:175}}
Apr 14 03:31:55 user nova-compute[71323]: WARNING os_brick.initiator.connectors.base [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:68}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:73}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[49905063-6509-4e9d-9e93-df4c20061dcd]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Apr 14 03:31:55 user nova-compute[71323]: No targets found.
Apr 14 03:31:55 user nova-compute[71323]: iSNS:
Apr 14 03:31:55 user nova-compute[71323]: No targets found.
Apr 14 03:31:55 user nova-compute[71323]: STATIC:
Apr 14 03:31:55 user nova-compute[71323]: Target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808
Apr 14 03:31:55 user nova-compute[71323]:         Portal: 172.16.0.220:3260,-1
Apr 14 03:31:55 user nova-compute[71323]:                 Iface Name: default
Apr 14 03:31:55 user nova-compute[71323]: FIRMWARE:
Apr 14 03:31:55 user nova-compute[71323]: No targets found.
Apr 14 03:31:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Apr 14 03:31:55 user nova-compute[71323]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Apr 14 03:31:55 user nova-compute[71323]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=71323) _get_discoverydb_portals /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:371}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808', 0)] {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:819}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updated VIF entry in instance network info cache for port a0610502-2adf-475c-822c-d96d457872a8. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.network.neutron [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [{"id": "a0610502-2adf-475c-822c-d96d457872a8", "address": "fa:16:3e:02:35:6f", "network": {"id": "f0e6e99b-d215-4622-9a1c-028eca82f6a0", "bridge": "br-int", "label": "tempest-TestShelveInstance-1037523706-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "1e0979cf1834435bb0007899da52e863", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa0610502-2a", "ovs_interfaceid": "a0610502-2adf-475c-822c-d96d457872a8", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[b32635b5-2d7c-4bf1-854f-daa84feb3a9d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-35d12e93-c1ed-4b3c-8b4f-9d0406925c7f req-68e85704-f56b-4c12-811a-adb1a5596dc0 service nova] Releasing lock "refresh_cache-ce889d46-fac9-4f77-9d66-6e7753d8a0e0" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c9c28a0-90fb-4274-87f6-e1515e1996eb]: (4, ('tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:31:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm_bare /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsi session list stdout=tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 (non-flash)
Apr 14 03:31:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsi_session /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f2b1cdca320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808'): ({'sdc'}, set())}) {{(pid=71323) _get_connection_devices /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:852}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Removing single pathed devices sdc {{(pid=71323) remove_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:309}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[18884c08-b906-4d2f-a956-0947a8b7a6ff]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.018s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[32368577-8086-46c4-8466-f60927fbf137]: (4, ('ok\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Flushing IO for device /dev/sdc {{(pid=71323) flush_device_io /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:369}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[61aad1d8-742f-4171-931a-1e14a550e541]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=71323) remove_scsi_device /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:83}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.038s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[0118062d-ee8b-47db-a418-4ffff60b0904]: (4, ('1', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:91}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.linuxscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] SCSI volumes sdc have been removed. {{(pid=71323) wait_for_volumes_removal /usr/local/lib/python3.10/dist-packages/os_brick/initiator/linuxscsi.py:101}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-23264633231623239', '/dev/disk/by-id/scsi-1SCST_BIOc9c9ae2a-2dc21b29a29e6fd4', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_c9c9ae2a') {{(pid=81992) unlink_root /usr/local/lib/python3.10/dist-packages/os_brick/privileged/rootwrap.py:219}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f94bb33-8843-4afc-9c26-4f19fbcf6b60]: (4, None) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808')] {{(pid=71323) _disconnect_connection /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1160}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e3649c4-d5c0-4be4-8896-fdfaf28acfb9]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --logout {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[78608ff2-e7f5-4c89-96d1-9694f2483580]: (4, ('Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]\nLogout of [sid: 53, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260]
Apr 14 03:31:55 user nova-compute[71323]: Logout of [sid: 53, target: iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808, portal: 172.16.0.220,3260] successful.
Apr 14 03:31:55 user nova-compute[71323]:  stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op delete {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=81992) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d7bf0bf-8ed3-4787-843d-59d6318dc254]: (4, ('', '')) {{(pid=81992) _call_back /usr/local/lib/python3.10/dist-packages/oslo_privsep/daemon.py:501}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=71323) _run_iscsiadm /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/iscsi.py:1026}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.base [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.293s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/os_brick/initiator/connectors/base.py:87}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG os_brick.initiator.connectors.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] <== disconnect_volume: return (294ms) None {{(pid=71323) trace_logging_wrapper /usr/local/lib/python3.10/dist-packages/os_brick/utils.py:202}}
Apr 14 03:31:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Disconnected iSCSI Volume {{(pid=71323) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:78}}
Apr 14 03:31:55 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deleting instance files /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0_del
Apr 14 03:31:55 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deletion of /opt/stack/data/nova/instances/ce889d46-fac9-4f77-9d66-6e7753d8a0e0_del complete
Apr 14 03:31:56 user nova-compute[71323]: INFO nova.compute.manager [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Took 1.17 seconds to destroy the instance on the hypervisor.
Apr 14 03:31:56 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:31:56 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:31:56 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:31:56 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:31:56 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Took 0.47 seconds to deallocate network for instance.
Apr 14 03:31:56 user nova-compute[71323]: DEBUG nova.compute.manager [req-0d3d860b-a37b-4bf2-9d5f-a978ca7464a7 req-f0e9bd79-6e79-4ce2-815e-e098ef5cbb8e service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-deleted-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] Acquiring lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] No waiting events found dispatching network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:31:57 user nova-compute[71323]: WARNING nova.compute.manager [req-3222deaf-8f35-46eb-8312-5eab902a9da9 req-0291e4d4-c670-485d-b4c3-19afce650d4d service nova] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Received unexpected event network-vif-plugged-a0610502-2adf-475c-822c-d96d457872a8 for instance with vm_state active and task_state deleting.
Apr 14 03:31:57 user nova-compute[71323]: INFO nova.compute.manager [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Took 0.74 seconds to detach 1 volumes for instance.
Apr 14 03:31:57 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Deleting volume: 3a8e1f1a-36b6-47b7-95f2-d31cd1b55808 {{(pid=71323) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3202}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.135s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:31:57 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Deleted allocations for instance ce889d46-fac9-4f77-9d66-6e7753d8a0e0
Apr 14 03:31:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-5cb79d5f-beb6-432f-99a5-2eeeb2a846cc tempest-TestShelveInstance-1966208901 tempest-TestShelveInstance-1966208901-project-member] Lock "ce889d46-fac9-4f77-9d66-6e7753d8a0e0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.756s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:07 user nova-compute[71323]: INFO nova.compute.manager [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Terminating instance
Apr 14 03:32:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:32:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-8c6119ed-eccc-4b3b-bbf2-e8e6b1956de2 req-24359ab9-a409-48c2-81a9-7d231b476fd2 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-unplugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:32:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Instance destroyed successfully.
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.objects.instance [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'resources' on Instance uuid 16e9b236-ce8f-4231-af55-e694bc43dde4 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:25:39Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-240447376',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-240447376',id=43,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJClBvq+izruhg+hEGBNF3dbytQyMHTMScVlN0JyjMZhWJs7Q0qW6oRUDMkWmHhWGirQ5OTk6+cGFOrE1Qs+cgpleFl1Ec3q//B2mJ5NTiLySOLz8iea1keakGG6m35P8g==',key_name='tempest-keypair-1931660222',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:29:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-1jbn1gar',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:29:44Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=16e9b236-ce8f-4231-af55-e694bc43dde4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "address": "fa:16:3e:11:d8:7b", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.68", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb47d2f30-8e", "ovs_interfaceid": "b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG os_vif [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb47d2f30-8e, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:08 user nova-compute[71323]: INFO os_vif [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:11:d8:7b,bridge_name='br-int',has_traffic_filtering=True,id=b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb47d2f30-8e')
Apr 14 03:32:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Deleting instance files /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4_del
Apr 14 03:32:08 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Deletion of /opt/stack/data/nova/instances/16e9b236-ce8f-4231-af55-e694bc43dde4_del complete
Apr 14 03:32:08 user nova-compute[71323]: INFO nova.compute.manager [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Took 0.83 seconds to destroy the instance on the hypervisor.
Apr 14 03:32:08 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:32:08 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:32:09 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Took 0.76 seconds to deallocate network for instance.
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.101s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:09 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Deleted allocations for instance 16e9b236-ce8f-4231-af55-e694bc43dde4
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-d611797d-55b2-468d-afe2-fc981be8f2cb tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.903s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] Acquiring lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] Lock "16e9b236-ce8f-4231-af55-e694bc43dde4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] No waiting events found dispatching network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:32:09 user nova-compute[71323]: WARNING nova.compute.manager [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received unexpected event network-vif-plugged-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 for instance with vm_state deleted and task_state None.
Apr 14 03:32:09 user nova-compute[71323]: DEBUG nova.compute.manager [req-571333fa-163d-40c4-9370-6f1acd7f8e22 req-d2bc62f5-74ce-4805-8fa5-c9f11f06dc73 service nova] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Received event network-vif-deleted-b47d2f30-8e02-4ff3-aae9-4b818e1ab8d4 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:32:10 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443115.6125677, ce889d46-fac9-4f77-9d66-6e7753d8a0e0 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:32:10 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] VM Stopped (Lifecycle Event)
Apr 14 03:32:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-bc0c08aa-bddf-4ea0-a2a6-6101d3c6c0b0 None None] [instance: ce889d46-fac9-4f77-9d66-6e7753d8a0e0] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:32:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:32:13 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:18 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:32:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:32:19 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8738MB free_disk=26.343894958496094GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:32:19 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.155s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:32:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:21 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:23 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443128.1930046, 16e9b236-ce8f-4231-af55-e694bc43dde4 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:32:23 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] VM Stopped (Lifecycle Event)
Apr 14 03:32:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-5622d364-3cb9-4f0b-b33f-f0503c4e1a33 None None] [instance: 16e9b236-ce8f-4231-af55-e694bc43dde4] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:32:23 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:32:25 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:32:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:28 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:32:33 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:32:38 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:48 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:51 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:53 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:32:58 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:33:01 user nova-compute[71323]: INFO nova.compute.claims [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Claim successful on node user
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.183s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:33:01 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:33:01 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.policy [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f88237b825b24effad17900bbf645ab8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ed26668c2cb49c38d16ff728d38bef2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:33:02 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Creating image(s)
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk 1073741824" returned: 0 in 0.084s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.234s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.126s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Checking if we can resize image /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Cannot resize image /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'migration_context' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Ensure instance console log exists: /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:02 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Successfully created port: c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Successfully updated port: c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG nova.compute.manager [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing instance network info cache due to event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:33:03 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.neutron [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance network_info: |[{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start _get_guest_xml network_info=[{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:33:04 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:33:04 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJfFuGWkFw8mYzv+IaggMRf+4kNylomilagNfTvTfzgsa+lN2O419d745uDaeH8a9DV6FeEnUxa7jF0bduYzwd6oy11A75oHtaHOK3aTQMv85AGBKT4RMQiCMXlR626PMg==',key_name='tempest-keypair-980674246',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:33:02Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.objects.instance [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_devices' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:33:04 user nova-compute[71323]:   <uuid>974938c0-18b8-47cd-8c37-4dbe6f228a53</uuid>
Apr 14 03:33:04 user nova-compute[71323]:   <name>instance-0000002e</name>
Apr 14 03:33:04 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:33:04 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:33:04 user nova-compute[71323]:   <metadata>
Apr 14 03:33:04 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:33:04 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1943005007</nova:name>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:33:04</nova:creationTime>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:33:04 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:33:04 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:owner>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:user uuid="f88237b825b24effad17900bbf645ab8">tempest-AttachVolumeShelveTestJSON-1546459049-project-member</nova:user>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:project uuid="7ed26668c2cb49c38d16ff728d38bef2">tempest-AttachVolumeShelveTestJSON-1546459049</nova:project>
Apr 14 03:33:04 user nova-compute[71323]:       </nova:owner>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:33:04 user nova-compute[71323]:       <nova:ports>
Apr 14 03:33:04 user nova-compute[71323]:         <nova:port uuid="c8eb45d0-1935-4985-97d4-4a73d14f204d">
Apr 14 03:33:04 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:33:04 user nova-compute[71323]:         </nova:port>
Apr 14 03:33:04 user nova-compute[71323]:       </nova:ports>
Apr 14 03:33:04 user nova-compute[71323]:     </nova:instance>
Apr 14 03:33:04 user nova-compute[71323]:   </metadata>
Apr 14 03:33:04 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:33:04 user nova-compute[71323]:     <system>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="serial">974938c0-18b8-47cd-8c37-4dbe6f228a53</entry>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="uuid">974938c0-18b8-47cd-8c37-4dbe6f228a53</entry>
Apr 14 03:33:04 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:33:04 user nova-compute[71323]:     </system>
Apr 14 03:33:04 user nova-compute[71323]:   </sysinfo>
Apr 14 03:33:04 user nova-compute[71323]:   <os>
Apr 14 03:33:04 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:33:04 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:33:04 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:33:04 user nova-compute[71323]:   </os>
Apr 14 03:33:04 user nova-compute[71323]:   <features>
Apr 14 03:33:04 user nova-compute[71323]:     <acpi/>
Apr 14 03:33:04 user nova-compute[71323]:     <apic/>
Apr 14 03:33:04 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:33:04 user nova-compute[71323]:   </features>
Apr 14 03:33:04 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:33:04 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:33:04 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:33:04 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:33:04 user nova-compute[71323]:   </clock>
Apr 14 03:33:04 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:33:04 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:33:04 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:33:04 user nova-compute[71323]:   </cpu>
Apr 14 03:33:04 user nova-compute[71323]:   <devices>
Apr 14 03:33:04 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:33:04 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:33:04 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk"/>
Apr 14 03:33:04 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:33:04 user nova-compute[71323]:     </disk>
Apr 14 03:33:04 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:33:04 user nova-compute[71323]:       <mac address="fa:16:3e:21:66:50"/>
Apr 14 03:33:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:33:04 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:33:04 user nova-compute[71323]:       <target dev="tapc8eb45d0-19"/>
Apr 14 03:33:04 user nova-compute[71323]:     </interface>
Apr 14 03:33:04 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:33:04 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/console.log" append="off"/>
Apr 14 03:33:04 user nova-compute[71323]:     </serial>
Apr 14 03:33:04 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:33:04 user nova-compute[71323]:     <video>
Apr 14 03:33:04 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:33:04 user nova-compute[71323]:     </video>
Apr 14 03:33:04 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:33:04 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:33:04 user nova-compute[71323]:     </rng>
Apr 14 03:33:04 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:33:04 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:33:04 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:33:04 user nova-compute[71323]:     </memballoon>
Apr 14 03:33:04 user nova-compute[71323]:   </devices>
Apr 14 03:33:04 user nova-compute[71323]: </domain>
Apr 14 03:33:04 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJfFuGWkFw8mYzv+IaggMRf+4kNylomilagNfTvTfzgsa+lN2O419d745uDaeH8a9DV6FeEnUxa7jF0bduYzwd6oy11A75oHtaHOK3aTQMv85AGBKT4RMQiCMXlR626PMg==',key_name='tempest-keypair-980674246',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:33:02Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG os_vif [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc8eb45d0-19, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc8eb45d0-19, col_values=(('external_ids', {'iface-id': 'c8eb45d0-1935-4985-97d4-4a73d14f204d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:21:66:50', 'vm-uuid': '974938c0-18b8-47cd-8c37-4dbe6f228a53'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:04 user nova-compute[71323]: INFO os_vif [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19')
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No VIF found with MAC fa:16:3e:21:66:50, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated VIF entry in instance network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG nova.network.neutron [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:33:04 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-dcfc7eea-1e64-49e6-a950-bc66c508aaba req-dea419a4-8a63-4302-adf3-62f3779f67fb service nova] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:05 user nova-compute[71323]: DEBUG nova.compute.manager [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:33:05 user nova-compute[71323]: WARNING nova.compute.manager [req-5d9b9af0-af7a-4bc0-a582-f4d26c5f5045 req-a49f6236-94da-4772-be3d-f15f18dd3515 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state building and task_state spawning.
Apr 14 03:33:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:06 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443187.6874068, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Resumed (Lifecycle Event)
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance spawned successfully.
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443187.688995, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Started (Lifecycle Event)
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Took 5.73 seconds to spawn the instance on the hypervisor.
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:07 user nova-compute[71323]: DEBUG nova.compute.manager [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:33:07 user nova-compute[71323]: WARNING nova.compute.manager [req-87f5c7c0-684f-4866-be14-174c3b4b2d92 req-5bd6b4d0-8073-426f-a718-1f23960ffd22 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state building and task_state spawning.
Apr 14 03:33:07 user nova-compute[71323]: INFO nova.compute.manager [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Took 6.23 seconds to build instance.
Apr 14 03:33:07 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-4af2f2c7-132b-4e80-8cdc-553425212884 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.326s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:12 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:33:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:33:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.150s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:33:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8748MB free_disk=26.31304168701172GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 974938c0-18b8-47cd-8c37-4dbe6f228a53 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:33:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.210s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:33:26 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:33:27 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:33:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:33:27 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:33:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_image_cache_manager_pass {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Verify base images {{(pid=71323) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:314}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Image id 98e6a247-250b-4bb5-a947-e7f93598d16e yields fingerprint 483d4346bff435b314742861dd205ae7be68253d {{(pid=71323) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] image 98e6a247-250b-4bb5-a947-e7f93598d16e at (/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d): checking
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] image 98e6a247-250b-4bb5-a947-e7f93598d16e at (/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d): image is in use {{(pid=71323) _mark_in_use /opt/stack/nova/nova/virt/libvirt/imagecache.py:279}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Image id  yields fingerprint da39a3ee5e6b4b0d3255bfef95601890afd80709 {{(pid=71323) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] 974938c0-18b8-47cd-8c37-4dbe6f228a53 is a valid instance name {{(pid=71323) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] 974938c0-18b8-47cd-8c37-4dbe6f228a53 has a disk file {{(pid=71323) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:129}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.144s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 974938c0-18b8-47cd-8c37-4dbe6f228a53 is backed by 483d4346bff435b314742861dd205ae7be68253d {{(pid=71323) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:141}}
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76
Apr 14 03:33:43 user nova-compute[71323]: WARNING nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Unknown base file: /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Active base files: /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Removable base files: /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3 /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784 /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755 /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76 /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/d163f2165f3760e6684a8f425d81d9c0fc425f9f
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/523c4ae7dc469830ffa38dc1f297039d8fb385f3
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/0286d9d01d1fe187421499cd39dcada15ca6c93e
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/e823b9712be850c2c32801ce101a377eb1abc784
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/2952a5f1d8e685bffd5079c6bcc24059df79fd6e
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/e2a0b14946e92e9e76c532f3d0a86d6addaca755
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/f6ce21d329050bd2327ddb777b2496513fa23e76
Apr 14 03:33:43 user nova-compute[71323]: INFO nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/1222e167d975bdd2cdbcf4c3b5ec71ebe0382a49
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Verification complete {{(pid=71323) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:350}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Verify swap images {{(pid=71323) _age_and_verify_swap_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:299}}
Apr 14 03:33:43 user nova-compute[71323]: DEBUG nova.virt.libvirt.imagecache [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Verify ephemeral images {{(pid=71323) _age_and_verify_ephemeral_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:284}}
Apr 14 03:33:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:33:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:10 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:10 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Triggering sync for uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10202}}
Apr 14 03:34:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:10 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.026s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:34:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:34:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:34:20 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.126s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:34:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:34:21 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8805MB free_disk=26.310718536376953GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 974938c0-18b8-47cd-8c37-4dbe6f228a53 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:34:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.351s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:34:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:26 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:34:28 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:36 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:34:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:47 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:34:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:34:52 user nova-compute[71323]: DEBUG nova.compute.manager [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing instance network info cache due to event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:34:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:34:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:34:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:34:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated VIF entry in instance network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:34:53 user nova-compute[71323]: DEBUG nova.network.neutron [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:34:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-d7f447bc-9a77-4d43-8918-3262940e6b74 req-0d1cf97a-21ad-4905-a85e-6503983bb39b service nova] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:34:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:34:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:57 user nova-compute[71323]: INFO nova.compute.manager [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Shelving
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance destroyed successfully.
Apr 14 03:34:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:34:57 user nova-compute[71323]: WARNING nova.compute.manager [req-7a76e69e-82eb-40d9-a2fb-7e587150015b req-8bdb95b9-bae3-4baf-88f4-5b4bc2cd4aac service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state active and task_state shelving.
Apr 14 03:34:57 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Beginning cold snapshot process
Apr 14 03:34:57 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk /opt/stack/data/nova/instances/snapshots/tmpnwc4gjqh/9830a60d3be4451683dc4394808caa48 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:34:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:58 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk /opt/stack/data/nova/instances/snapshots/tmpnwc4gjqh/9830a60d3be4451683dc4394808caa48" returned: 0 in 0.464s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:34:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Snapshot extracted, beginning image upload
Apr 14 03:34:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:34:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:34:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:34:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:34:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:34:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:34:59 user nova-compute[71323]: WARNING nova.compute.manager [req-90a2475b-4c1f-4f5c-bb99-6dc02949816e req-29468ff1-6c77-4951-9f5f-f569a5a34d90 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state active and task_state shelving_image_uploading.
Apr 14 03:34:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Snapshot image upload complete
Apr 14 03:35:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:35:00 user nova-compute[71323]: INFO nova.compute.manager [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Shelve offloading
Apr 14 03:35:00 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance destroyed successfully.
Apr 14 03:35:00 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:35:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:35:00 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:35:00 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:35:01 user nova-compute[71323]: DEBUG nova.network.neutron [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:35:01 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:35:02 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance destroyed successfully.
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'resources' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJfFuGWkFw8mYzv+IaggMRf+4kNylomilagNfTvTfzgsa+lN2O419d745uDaeH8a9DV6FeEnUxa7jF0bduYzwd6oy11A75oHtaHOK3aTQMv85AGBKT4RMQiCMXlR626PMg==',key_name='tempest-keypair-980674246',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:33:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:35:00.721841',shelved_host='user',shelved_image_id='606f202d-669c-412c-85ec-7df354098ce0'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:34:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG os_vif [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc8eb45d0-19, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.compute.manager [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.compute.manager [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing instance network info cache due to event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.network.neutron [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:35:02 user nova-compute[71323]: INFO os_vif [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19')
Apr 14 03:35:02 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Deleting instance files /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53_del
Apr 14 03:35:02 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Deletion of /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53_del complete
Apr 14 03:35:02 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Deleted allocations for instance 974938c0-18b8-47cd-8c37-4dbe6f228a53
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.objects.instance [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'numa_topology' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.099s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:35:02 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-0abd59c6-5c8c-40ac-bb65-2bc0c9555be3 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 5.874s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:35:03 user nova-compute[71323]: DEBUG nova.network.neutron [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated VIF entry in instance network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:35:03 user nova-compute[71323]: DEBUG nova.network.neutron [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:35:03 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-4976c4cf-36b3-40ef-a285-0171e972e57e req-78e5e36c-105b-4be9-a08b-73feee5f662f service nova] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:35:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:12 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443297.6108396, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:35:12 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Stopped (Lifecycle Event)
Apr 14 03:35:12 user nova-compute[71323]: DEBUG nova.compute.manager [None req-e2d5cbb8-6320-4a70-813f-35da034f77fb None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:35:13 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:14 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:35:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:35:21 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:35:22 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:35:22 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8867MB free_disk=26.293506622314453GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.160s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:35:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:35:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:35:28 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:35:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:35:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:35:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:35:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:36:07 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:12 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:14 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:36:17 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:21 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:36:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:36:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8897MB free_disk=26.292957305908203GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:36:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.257s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:24 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:27 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:36:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:36:30 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:36:32 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:33 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:36:37 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:42 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:47 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:49 user nova-compute[71323]: INFO nova.compute.manager [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Unshelving
Apr 14 03:36:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_requests' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'numa_topology' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:36:49 user nova-compute[71323]: INFO nova.compute.claims [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Claim successful on node user
Apr 14 03:36:49 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:36:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.213s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:49 user nova-compute[71323]: INFO nova.network.neutron [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating port c8eb45d0-1935-4985-97d4-4a73d14f204d with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing instance network info cache due to event network-changed-c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:36:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG nova.network.neutron [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:36:51 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Creating image(s)
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'trusted_certs' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG nova.network.neutron [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Refreshing network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "e8e638709a38e40e42925bef9fd4b500566aa36a" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:51 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "e8e638709a38e40e42925bef9fd4b500566aa36a" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.part --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.part --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG nova.virt.images [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] 606f202d-669c-412c-85ec-7df354098ce0 was qcow2, converting to raw {{(pid=71323) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG nova.privsep.utils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=71323) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.part /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.converted {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated VIF entry in instance network info cache for port c8eb45d0-1935-4985-97d4-4a73d14f204d. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG nova.network.neutron [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-9f231ad9-cbaa-4ce4-a9e3-8624c6a071ef req-25da014c-370f-4468-8a5f-17c9ab83bbf4 service nova] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.part /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.converted" returned: 0 in 0.436s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.converted --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a.converted --force-share --output=json" returned: 0 in 0.133s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "e8e638709a38e40e42925bef9fd4b500566aa36a" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.736s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json" returned: 0 in 0.128s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "e8e638709a38e40e42925bef9fd4b500566aa36a" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "e8e638709a38e40e42925bef9fd4b500566aa36a" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:52 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json" returned: 0 in 0.134s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a,backing_fmt=raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a,backing_fmt=raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk 1073741824" returned: 0 in 0.056s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "e8e638709a38e40e42925bef9fd4b500566aa36a" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.197s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/e8e638709a38e40e42925bef9fd4b500566aa36a --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'migration_context' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:53 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Rebasing disk image.
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:53 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d -F raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d -F raw /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk" returned: 0 in 1.861s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Ensure instance console log exists: /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start _get_guest_xml network_info=[{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d517a771730fa81fb453faf8cafad548',container_format='bare',created_at=2023-04-14T03:34:57Z,direct_url=<?>,disk_format='qcow2',id=606f202d-669c-412c-85ec-7df354098ce0,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1943005007-shelved',owner='7ed26668c2cb49c38d16ff728d38bef2',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T03:35:01Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:36:55 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:36:55 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d517a771730fa81fb453faf8cafad548',container_format='bare',created_at=2023-04-14T03:34:57Z,direct_url=<?>,disk_format='qcow2',id=606f202d-669c-412c-85ec-7df354098ce0,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-1943005007-shelved',owner='7ed26668c2cb49c38d16ff728d38bef2',properties=ImageMetaProps,protected=<?>,size=38600704,status='active',tags=<?>,updated_at=2023-04-14T03:35:01Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'vcpu_model' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='606f202d-669c-412c-85ec-7df354098ce0',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-980674246',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:33:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:35:00.721841',shelved_host='user',shelved_image_id='606f202d-669c-412c-85ec-7df354098ce0'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:36:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.objects.instance [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'pci_devices' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:36:55 user nova-compute[71323]:   <uuid>974938c0-18b8-47cd-8c37-4dbe6f228a53</uuid>
Apr 14 03:36:55 user nova-compute[71323]:   <name>instance-0000002e</name>
Apr 14 03:36:55 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:36:55 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:36:55 user nova-compute[71323]:   <metadata>
Apr 14 03:36:55 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:36:55 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-1943005007</nova:name>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:36:55</nova:creationTime>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:36:55 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:36:55 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:owner>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:user uuid="f88237b825b24effad17900bbf645ab8">tempest-AttachVolumeShelveTestJSON-1546459049-project-member</nova:user>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:project uuid="7ed26668c2cb49c38d16ff728d38bef2">tempest-AttachVolumeShelveTestJSON-1546459049</nova:project>
Apr 14 03:36:55 user nova-compute[71323]:       </nova:owner>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:root type="image" uuid="606f202d-669c-412c-85ec-7df354098ce0"/>
Apr 14 03:36:55 user nova-compute[71323]:       <nova:ports>
Apr 14 03:36:55 user nova-compute[71323]:         <nova:port uuid="c8eb45d0-1935-4985-97d4-4a73d14f204d">
Apr 14 03:36:55 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Apr 14 03:36:55 user nova-compute[71323]:         </nova:port>
Apr 14 03:36:55 user nova-compute[71323]:       </nova:ports>
Apr 14 03:36:55 user nova-compute[71323]:     </nova:instance>
Apr 14 03:36:55 user nova-compute[71323]:   </metadata>
Apr 14 03:36:55 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:36:55 user nova-compute[71323]:     <system>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="serial">974938c0-18b8-47cd-8c37-4dbe6f228a53</entry>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="uuid">974938c0-18b8-47cd-8c37-4dbe6f228a53</entry>
Apr 14 03:36:55 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:36:55 user nova-compute[71323]:     </system>
Apr 14 03:36:55 user nova-compute[71323]:   </sysinfo>
Apr 14 03:36:55 user nova-compute[71323]:   <os>
Apr 14 03:36:55 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:36:55 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:36:55 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:36:55 user nova-compute[71323]:   </os>
Apr 14 03:36:55 user nova-compute[71323]:   <features>
Apr 14 03:36:55 user nova-compute[71323]:     <acpi/>
Apr 14 03:36:55 user nova-compute[71323]:     <apic/>
Apr 14 03:36:55 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:36:55 user nova-compute[71323]:   </features>
Apr 14 03:36:55 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:36:55 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:36:55 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:36:55 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:36:55 user nova-compute[71323]:   </clock>
Apr 14 03:36:55 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:36:55 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:36:55 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:36:55 user nova-compute[71323]:   </cpu>
Apr 14 03:36:55 user nova-compute[71323]:   <devices>
Apr 14 03:36:55 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:36:55 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:36:55 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk"/>
Apr 14 03:36:55 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:36:55 user nova-compute[71323]:     </disk>
Apr 14 03:36:55 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:36:55 user nova-compute[71323]:       <mac address="fa:16:3e:21:66:50"/>
Apr 14 03:36:55 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:36:55 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:36:55 user nova-compute[71323]:       <target dev="tapc8eb45d0-19"/>
Apr 14 03:36:55 user nova-compute[71323]:     </interface>
Apr 14 03:36:55 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:36:55 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/console.log" append="off"/>
Apr 14 03:36:55 user nova-compute[71323]:     </serial>
Apr 14 03:36:55 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:36:55 user nova-compute[71323]:     <video>
Apr 14 03:36:55 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:36:55 user nova-compute[71323]:     </video>
Apr 14 03:36:55 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:36:55 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:36:55 user nova-compute[71323]:     </rng>
Apr 14 03:36:55 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:36:55 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:36:55 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:36:55 user nova-compute[71323]:     </memballoon>
Apr 14 03:36:55 user nova-compute[71323]:   </devices>
Apr 14 03:36:55 user nova-compute[71323]: </domain>
Apr 14 03:36:55 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='606f202d-669c-412c-85ec-7df354098ce0',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name='tempest-keypair-980674246',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:33:07Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member',shelved_at='2023-04-14T03:35:00.721841',shelved_host='user',shelved_image_id='606f202d-669c-412c-85ec-7df354098ce0'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:36:50Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG os_vif [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc8eb45d0-19, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tapc8eb45d0-19, col_values=(('external_ids', {'iface-id': 'c8eb45d0-1935-4985-97d4-4a73d14f204d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:21:66:50', 'vm-uuid': '974938c0-18b8-47cd-8c37-4dbe6f228a53'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:55 user nova-compute[71323]: INFO os_vif [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19')
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:36:55 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] No VIF found with MAC fa:16:3e:21:66:50, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:36:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:56 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG nova.compute.manager [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:36:57 user nova-compute[71323]: WARNING nova.compute.manager [req-b9cd50fb-56ac-48bb-bfd8-68f582db6291 req-921b6fdf-45a8-4a7b-a573-e589d7fb9b88 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 03:36:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:57 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443418.8544092, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:36:58 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Resumed (Lifecycle Event)
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:36:58 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance spawned successfully.
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:36:58 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443418.8580465, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:36:58 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Started (Lifecycle Event)
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:36:58 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Synchronizing instance power state after lifecycle event "Started"; current vm_state: shelved_offloaded, current task_state: spawning, current DB power_state: 4, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:36:58 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:36:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG nova.compute.manager [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:36:59 user nova-compute[71323]: WARNING nova.compute.manager [req-21061379-a672-480c-969b-ca06c6b9f4df req-6b3f09de-5b4d-479e-bf13-c8d50382af5b service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state shelved_offloaded and task_state spawning.
Apr 14 03:36:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-3e27a45f-188f-42b2-8250-5761b01bf6c7 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 9.589s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:36:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:16 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:37:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:37:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:37:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:37:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8815MB free_disk=26.27458953857422GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 974938c0-18b8-47cd-8c37-4dbe6f228a53 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:37:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.208s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:37:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:37:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:37:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:37:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:37:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:37:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5004 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:37:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:38:22 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53/disk --force-share --output=json" returned: 0 in 0.142s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:38:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:38:23 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8795MB free_disk=26.273998260498047GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 974938c0-18b8-47cd-8c37-4dbe6f228a53 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:38:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.155s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:38:32 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:38:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [{"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:38:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-974938c0-18b8-47cd-8c37-4dbe6f228a53" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:38:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:38:33 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:39 user nova-compute[71323]: INFO nova.compute.manager [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Terminating instance
Apr 14 03:38:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.compute.manager [req-5495cd78-db23-486e-8534-04fa46aa213e req-0e9ef9f7-3b1c-4fc9-8f96-e19d742a5094 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-unplugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Instance destroyed successfully.
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lazy-loading 'resources' on Instance uuid 974938c0-18b8-47cd-8c37-4dbe6f228a53 {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2023-04-14T03:33:01Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-1943005007',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-1943005007',id=46,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJfFuGWkFw8mYzv+IaggMRf+4kNylomilagNfTvTfzgsa+lN2O419d745uDaeH8a9DV6FeEnUxa7jF0bduYzwd6oy11A75oHtaHOK3aTQMv85AGBKT4RMQiCMXlR626PMg==',key_name='tempest-keypair-980674246',keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:36:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='7ed26668c2cb49c38d16ff728d38bef2',ramdisk_id='',reservation_id='r-ocqzi5uh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1546459049',owner_user_name='tempest-AttachVolumeShelveTestJSON-1546459049-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:36:59Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f88237b825b24effad17900bbf645ab8',uuid=974938c0-18b8-47cd-8c37-4dbe6f228a53,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converting VIF {"id": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "address": "fa:16:3e:21:66:50", "network": {"id": "37d06f06-d5c9-414b-bbc2-17c9a433bc16", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-697579486-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.42", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7ed26668c2cb49c38d16ff728d38bef2", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8eb45d0-19", "ovs_interfaceid": "c8eb45d0-1935-4985-97d4-4a73d14f204d", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG os_vif [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc8eb45d0-19, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:40 user nova-compute[71323]: INFO os_vif [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:21:66:50,bridge_name='br-int',has_traffic_filtering=True,id=c8eb45d0-1935-4985-97d4-4a73d14f204d,network=Network(37d06f06-d5c9-414b-bbc2-17c9a433bc16),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8eb45d0-19')
Apr 14 03:38:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Deleting instance files /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53_del
Apr 14 03:38:40 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Deletion of /opt/stack/data/nova/instances/974938c0-18b8-47cd-8c37-4dbe6f228a53_del complete
Apr 14 03:38:40 user nova-compute[71323]: INFO nova.compute.manager [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Took 0.91 seconds to destroy the instance on the hypervisor.
Apr 14 03:38:40 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:38:40 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:38:41 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Took 0.88 seconds to deallocate network for instance.
Apr 14 03:38:41 user nova-compute[71323]: DEBUG nova.compute.manager [req-32ea3447-76da-4007-a6bb-1e7e8926dfc8 req-5c36ce94-81e3-4dcd-bf0d-7da4bfd0fbc8 service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-deleted-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:38:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.204s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:41 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Deleted allocations for instance 974938c0-18b8-47cd-8c37-4dbe6f228a53
Apr 14 03:38:41 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c0bb5e01-8546-40f4-b040-9064f791af42 tempest-AttachVolumeShelveTestJSON-1546459049 tempest-AttachVolumeShelveTestJSON-1546459049-project-member] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.185s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:38:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] Acquiring lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:38:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:38:42 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] Lock "974938c0-18b8-47cd-8c37-4dbe6f228a53-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:38:42 user nova-compute[71323]: DEBUG nova.compute.manager [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] No waiting events found dispatching network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:38:42 user nova-compute[71323]: WARNING nova.compute.manager [req-ede3163b-965c-4f3f-856a-c86f6b90ebb7 req-db5f4efe-51f9-466f-951b-df90bcf67ade service nova] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Received unexpected event network-vif-plugged-c8eb45d0-1935-4985-97d4-4a73d14f204d for instance with vm_state deleted and task_state None.
Apr 14 03:38:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:38:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443520.4685984, 974938c0-18b8-47cd-8c37-4dbe6f228a53 => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:38:55 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] VM Stopped (Lifecycle Event)
Apr 14 03:38:55 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7406d3a6-6538-4eec-a233-4895bab465be None None] [instance: 974938c0-18b8-47cd-8c37-4dbe6f228a53] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4998-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:38:55 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:00 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:05 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:39:10 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:15 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:15 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:18 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:39:20 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:23 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:23 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:39:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:39:24 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8866MB free_disk=26.29308319091797GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:39:24 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.264s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:25 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:30 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:33 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:39:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:39:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:39:33 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:35 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:39 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:39 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances with incomplete migration  {{(pid=71323) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11117}}
Apr 14 03:39:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:40 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:44 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Cleaning up deleted instances {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11079}}
Apr 14 03:39:44 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] There are 0 instances to clean {{(pid=71323) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11088}}
Apr 14 03:39:45 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Starting instance... {{(pid=71323) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2404}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=71323) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2338}}
Apr 14 03:39:46 user nova-compute[71323]: INFO nova.compute.claims [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Claim successful on node user
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.192s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Start building networks asynchronously for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2784}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Allocating IP information in the background. {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1957}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] allocate_for_instance() {{(pid=71323) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1154}}
Apr 14 03:39:46 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Start building block device mappings for instance. {{(pid=71323) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Apr 14 03:39:46 user nova-compute[71323]: DEBUG nova.policy [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '2975e8c90d3e41dfa099c98f41635950', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '778eb822f6cb4f9c9c388ee7ed8a2071', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=71323) authorize /opt/stack/nova/nova/policy.py:203}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Start spawning the instance on the hypervisor. {{(pid=71323) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2604}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Creating instance directory {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4698}}
Apr 14 03:39:47 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Creating image(s)
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "/opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "/opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "/opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.149s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "483d4346bff435b314742861dd205ae7be68253d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.137s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk 1073741824 {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d,backing_fmt=raw /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk 1073741824" returned: 0 in 0.047s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "483d4346bff435b314742861dd205ae7be68253d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.191s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/483d4346bff435b314742861dd205ae7be68253d --force-share --output=json" returned: 0 in 0.132s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Checking if we can resize image /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk. size=1073741824 {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Successfully created port: 8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:546}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json" returned: 0 in 0.138s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.virt.disk.api [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Cannot resize image /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk to a smaller size. {{(pid=71323) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lazy-loading 'migration_context' on Instance uuid 536a066d-074b-4a88-94d2-9815421b70da {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Created local disks {{(pid=71323) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4832}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Ensure instance console log exists: /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/console.log {{(pid=71323) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4584}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:47 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Successfully updated port: 8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) _update_port /opt/stack/nova/nova/network/neutron.py:584}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquired lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Building network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2000}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-changed-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG nova.compute.manager [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Refreshing instance network info cache due to event network-changed-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17. {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10987}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] Acquiring lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:39:48 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Instance cache missing network info. {{(pid=71323) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3313}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.neutron [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updating instance_info_cache with network_info: [{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Releasing lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Instance network_info: |[{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=71323) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] Acquired lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Refreshing network info cache for port 8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1997}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Start _get_guest_xml network_info=[{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_type': 'disk', 'encrypted': False, 'encryption_format': None, 'disk_bus': 'virtio', 'guest_format': None, 'device_name': '/dev/vda', 'boot_index': 0, 'encryption_options': None, 'size': 0, 'encryption_secret_uuid': None, 'image_id': '98e6a247-250b-4bb5-a947-e7f93598d16e'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7526}}
Apr 14 03:39:49 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:39:49 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=71323) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5371}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-04-14T02:55:17Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='42',id=11,is_public=True,memory_mb=128,name='m1.nano',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='b874c39491a2377b8490f5f1e89761a4',container_format='bare',created_at=2023-04-14T02:54:11Z,direct_url=<?>,disk_format='qcow2',id=98e6a247-250b-4bb5-a947-e7f93598d16e,min_disk=0,min_ram=0,name='cirros-0.5.2-x86_64-disk',owner='172ef384b95f49dc8d6e8f73eb7dae92',properties=ImageMetaProps,protected=<?>,size=16300544,status='active',tags=<?>,updated_at=2023-04-14T02:54:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:558}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Flavor limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Image limits 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Flavor pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Image pref 0:0:0 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=71323) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:564}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:466}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Got 1 possible topologies {{(pid=71323) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:496}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:570}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.hardware [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=71323) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:572}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-712565026',display_name='tempest-VolumesActionsTest-instance-712565026',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-712565026',id=47,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='778eb822f6cb4f9c9c388ee7ed8a2071',ramdisk_id='',reservation_id='r-f1xlak59',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-376578855',owner_user_name='tempest-VolumesActionsTest-376578855-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:39:47Z,user_data=None,user_id='2975e8c90d3e41dfa099c98f41635950',uuid=536a066d-074b-4a88-94d2-9815421b70da,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=71323) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converting VIF {"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.objects.instance [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lazy-loading 'pci_devices' on Instance uuid 536a066d-074b-4a88-94d2-9815421b70da {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] End _get_guest_xml xml=<domain type="kvm">
Apr 14 03:39:49 user nova-compute[71323]:   <uuid>536a066d-074b-4a88-94d2-9815421b70da</uuid>
Apr 14 03:39:49 user nova-compute[71323]:   <name>instance-0000002f</name>
Apr 14 03:39:49 user nova-compute[71323]:   <memory>131072</memory>
Apr 14 03:39:49 user nova-compute[71323]:   <vcpu>1</vcpu>
Apr 14 03:39:49 user nova-compute[71323]:   <metadata>
Apr 14 03:39:49 user nova-compute[71323]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Apr 14 03:39:49 user nova-compute[71323]:       <nova:package version="0.0.0"/>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:name>tempest-VolumesActionsTest-instance-712565026</nova:name>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:creationTime>2023-04-14 03:39:49</nova:creationTime>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:flavor name="m1.nano">
Apr 14 03:39:49 user nova-compute[71323]:         <nova:memory>128</nova:memory>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:disk>1</nova:disk>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:swap>0</nova:swap>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:ephemeral>0</nova:ephemeral>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:vcpus>1</nova:vcpus>
Apr 14 03:39:49 user nova-compute[71323]:       </nova:flavor>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:owner>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:user uuid="2975e8c90d3e41dfa099c98f41635950">tempest-VolumesActionsTest-376578855-project-member</nova:user>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:project uuid="778eb822f6cb4f9c9c388ee7ed8a2071">tempest-VolumesActionsTest-376578855</nova:project>
Apr 14 03:39:49 user nova-compute[71323]:       </nova:owner>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:root type="image" uuid="98e6a247-250b-4bb5-a947-e7f93598d16e"/>
Apr 14 03:39:49 user nova-compute[71323]:       <nova:ports>
Apr 14 03:39:49 user nova-compute[71323]:         <nova:port uuid="8c7a028d-cb6f-4cc7-8469-07fb8c5afe17">
Apr 14 03:39:49 user nova-compute[71323]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Apr 14 03:39:49 user nova-compute[71323]:         </nova:port>
Apr 14 03:39:49 user nova-compute[71323]:       </nova:ports>
Apr 14 03:39:49 user nova-compute[71323]:     </nova:instance>
Apr 14 03:39:49 user nova-compute[71323]:   </metadata>
Apr 14 03:39:49 user nova-compute[71323]:   <sysinfo type="smbios">
Apr 14 03:39:49 user nova-compute[71323]:     <system>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="manufacturer">OpenStack Foundation</entry>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="product">OpenStack Nova</entry>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="version">0.0.0</entry>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="serial">536a066d-074b-4a88-94d2-9815421b70da</entry>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="uuid">536a066d-074b-4a88-94d2-9815421b70da</entry>
Apr 14 03:39:49 user nova-compute[71323]:       <entry name="family">Virtual Machine</entry>
Apr 14 03:39:49 user nova-compute[71323]:     </system>
Apr 14 03:39:49 user nova-compute[71323]:   </sysinfo>
Apr 14 03:39:49 user nova-compute[71323]:   <os>
Apr 14 03:39:49 user nova-compute[71323]:     <type arch="x86_64" machine="pc">hvm</type>
Apr 14 03:39:49 user nova-compute[71323]:     <boot dev="hd"/>
Apr 14 03:39:49 user nova-compute[71323]:     <smbios mode="sysinfo"/>
Apr 14 03:39:49 user nova-compute[71323]:   </os>
Apr 14 03:39:49 user nova-compute[71323]:   <features>
Apr 14 03:39:49 user nova-compute[71323]:     <acpi/>
Apr 14 03:39:49 user nova-compute[71323]:     <apic/>
Apr 14 03:39:49 user nova-compute[71323]:     <vmcoreinfo/>
Apr 14 03:39:49 user nova-compute[71323]:   </features>
Apr 14 03:39:49 user nova-compute[71323]:   <clock offset="utc">
Apr 14 03:39:49 user nova-compute[71323]:     <timer name="pit" tickpolicy="delay"/>
Apr 14 03:39:49 user nova-compute[71323]:     <timer name="rtc" tickpolicy="catchup"/>
Apr 14 03:39:49 user nova-compute[71323]:     <timer name="hpet" present="no"/>
Apr 14 03:39:49 user nova-compute[71323]:   </clock>
Apr 14 03:39:49 user nova-compute[71323]:   <cpu mode="custom" match="exact">
Apr 14 03:39:49 user nova-compute[71323]:     <model>Nehalem</model>
Apr 14 03:39:49 user nova-compute[71323]:     <topology sockets="1" cores="1" threads="1"/>
Apr 14 03:39:49 user nova-compute[71323]:   </cpu>
Apr 14 03:39:49 user nova-compute[71323]:   <devices>
Apr 14 03:39:49 user nova-compute[71323]:     <disk type="file" device="disk">
Apr 14 03:39:49 user nova-compute[71323]:       <driver name="qemu" type="qcow2" cache="none"/>
Apr 14 03:39:49 user nova-compute[71323]:       <source file="/opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk"/>
Apr 14 03:39:49 user nova-compute[71323]:       <target dev="vda" bus="virtio"/>
Apr 14 03:39:49 user nova-compute[71323]:     </disk>
Apr 14 03:39:49 user nova-compute[71323]:     <interface type="ethernet">
Apr 14 03:39:49 user nova-compute[71323]:       <mac address="fa:16:3e:f6:c7:c0"/>
Apr 14 03:39:49 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:39:49 user nova-compute[71323]:       <mtu size="1442"/>
Apr 14 03:39:49 user nova-compute[71323]:       <target dev="tap8c7a028d-cb"/>
Apr 14 03:39:49 user nova-compute[71323]:     </interface>
Apr 14 03:39:49 user nova-compute[71323]:     <serial type="pty">
Apr 14 03:39:49 user nova-compute[71323]:       <log file="/opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/console.log" append="off"/>
Apr 14 03:39:49 user nova-compute[71323]:     </serial>
Apr 14 03:39:49 user nova-compute[71323]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Apr 14 03:39:49 user nova-compute[71323]:     <video>
Apr 14 03:39:49 user nova-compute[71323]:       <model type="virtio"/>
Apr 14 03:39:49 user nova-compute[71323]:     </video>
Apr 14 03:39:49 user nova-compute[71323]:     <rng model="virtio">
Apr 14 03:39:49 user nova-compute[71323]:       <backend model="random">/dev/urandom</backend>
Apr 14 03:39:49 user nova-compute[71323]:     </rng>
Apr 14 03:39:49 user nova-compute[71323]:     <controller type="usb" index="0" model="none"/>
Apr 14 03:39:49 user nova-compute[71323]:     <memballoon model="virtio">
Apr 14 03:39:49 user nova-compute[71323]:       <stats period="10"/>
Apr 14 03:39:49 user nova-compute[71323]:     </memballoon>
Apr 14 03:39:49 user nova-compute[71323]:   </devices>
Apr 14 03:39:49 user nova-compute[71323]: </domain>
Apr 14 03:39:49 user nova-compute[71323]:  {{(pid=71323) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7532}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-712565026',display_name='tempest-VolumesActionsTest-instance-712565026',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-712565026',id=47,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='778eb822f6cb4f9c9c388ee7ed8a2071',ramdisk_id='',reservation_id='r-f1xlak59',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-376578855',owner_user_name='tempest-VolumesActionsTest-376578855-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-04-14T03:39:47Z,user_data=None,user_id='2975e8c90d3e41dfa099c98f41635950',uuid=536a066d-074b-4a88-94d2-9815421b70da,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converting VIF {"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG os_vif [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb') {{(pid=71323) plug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:76}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8c7a028d-cb, may_exist=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Interface, record=tap8c7a028d-cb, col_values=(('external_ids', {'iface-id': '8c7a028d-cb6f-4cc7-8469-07fb8c5afe17', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f6:c7:c0', 'vm-uuid': '536a066d-074b-4a88-94d2-9815421b70da'}),)) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:49 user nova-compute[71323]: INFO os_vif [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb')
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] No BDM found with device name vda, not building metadata. {{(pid=71323) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] No VIF found with MAC fa:16:3e:f6:c7:c0, not building metadata {{(pid=71323) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12041}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updated VIF entry in instance network info cache for port 8c7a028d-cb6f-4cc7-8469-07fb8c5afe17. {{(pid=71323) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3472}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG nova.network.neutron [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updating instance_info_cache with network_info: [{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:39:49 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-624e9842-592a-4525-9045-a4f04739fa48 req-b5242480-f165-4dc9-8b1c-86e887790bce service nova] Releasing lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:50 user nova-compute[71323]: DEBUG nova.compute.manager [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] No waiting events found dispatching network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:39:50 user nova-compute[71323]: WARNING nova.compute.manager [req-6f06c064-cc0b-47da-9265-a11fb6b85ecb req-006c8712-96c4-46a8-a399-457a638caa8a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received unexpected event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 for instance with vm_state building and task_state spawning.
Apr 14 03:39:52 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443592.7767117, 536a066d-074b-4a88-94d2-9815421b70da => Resumed> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] VM Resumed (Lifecycle Event)
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Instance event wait completed in 0 seconds for  {{(pid=71323) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Guest created on hypervisor {{(pid=71323) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4392}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Instance spawned successfully.
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:889}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Synchronizing instance power state after lifecycle event "Resumed"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_cdrom_bus of ide {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_disk_bus of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_input_bus of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_pointer_model of None {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_video_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.libvirt.driver [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Found default for hw_vif_model of virtio {{(pid=71323) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:918}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.virt.driver [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] Emitting event <LifecycleEvent: 1681443592.7801208, 536a066d-074b-4a88-94d2-9815421b70da => Started> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] VM Started (Lifecycle Event)
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 {{(pid=71323) handle_lifecycle_event /opt/stack/nova/nova/compute/manager.py:1396}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-0390372e-1769-4ab2-a7b4-02ea8e110ef1 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] During sync_power_state the instance has a pending task (spawning). Skip.
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Took 5.89 seconds to spawn the instance on the hypervisor.
Apr 14 03:39:52 user nova-compute[71323]: DEBUG nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:39:52 user nova-compute[71323]: INFO nova.compute.manager [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Took 6.39 seconds to build instance.
Apr 14 03:39:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-c29e92ed-0306-4b0d-a445-7a7a47e80653 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.489s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:39:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:39:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:39:53 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:39:53 user nova-compute[71323]: DEBUG nova.compute.manager [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] No waiting events found dispatching network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:39:53 user nova-compute[71323]: WARNING nova.compute.manager [req-07bfa8ab-fd4d-4993-8fa8-9f350745185c req-617f4279-8aac-4c70-bff8-9310c749e101 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received unexpected event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 for instance with vm_state active and task_state None.
Apr 14 03:39:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:39:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:19 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:19 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:40:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json" returned: 0 in 0.136s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:40:25 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json" returned: 0 in 0.143s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:40:26 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:40:26 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8807MB free_disk=26.272289276123047GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 536a066d-074b-4a88-94d2-9815421b70da actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:40:26 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.176s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:40:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:29 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:40:33 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 536a066d-074b-4a88-94d2-9815421b70da {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updating instance_info_cache with network_info: [{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:40:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:35 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:41 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:40:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:41:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:26 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:41:27 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json" returned: 0 in 0.130s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running cmd (subprocess): /usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:384}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.processutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CMD "/usr/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da/disk --force-share --output=json" returned: 0 in 0.139s {{(pid=71323) execute /usr/local/lib/python3.10/dist-packages/oslo_concurrency/processutils.py:422}}
Apr 14 03:41:28 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:41:28 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8811MB free_disk=26.27170181274414GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Instance 536a066d-074b-4a88-94d2-9815421b70da actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 128, 'VCPU': 1}}. {{(pid=71323) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1632}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=640MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing inventories for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating ProviderTree inventory for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Updating inventory in ProviderTree for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing aggregate associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, aggregates: None {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Refreshing trait associations for resource provider 8eb2236f-d4b5-4afe-871b-37575055ded0, traits: COMPUTE_VOLUME_EXTEND,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,HW_CPU_X86_SSE42,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_STORAGE_BUS_SATA,COMPUTE_GRAPHICS_MODEL_QXL,HW_CPU_X86_SSE41,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_FDC,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_NODE,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_GRAPHICS_MODEL_VIRTIO,HW_CPU_X86_SSE2,COMPUTE_IMAGE_TYPE_AKI,HW_CPU_X86_SSE,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_NET_VIF_MODEL_RTL8139,COMPUTE_RESCUE_BFV,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_STORAGE_BUS_USB,COMPUTE_ACCELERATORS,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,HW_CPU_X86_SSSE3,COMPUTE_STORAGE_BUS_IDE,COMPUTE_VOLUME_ATTACH_WITH_TAG,HW_CPU_X86_MMX {{(pid=71323) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:41:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.285s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquired lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Forcefully refreshing network info cache for instance {{(pid=71323) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1994}}
Apr 14 03:41:35 user nova-compute[71323]: DEBUG nova.objects.instance [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lazy-loading 'info_cache' on Instance uuid 536a066d-074b-4a88-94d2-9815421b70da {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:41:36 user nova-compute[71323]: DEBUG nova.network.neutron [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updating instance_info_cache with network_info: [{"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:41:36 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Releasing lock "refresh_cache-536a066d-074b-4a88-94d2-9815421b70da" {{(pid=71323) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}}
Apr 14 03:41:36 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updated the network info_cache for instance {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9863}}
Apr 14 03:41:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:41:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:43 user nova-compute[71323]: INFO nova.compute.manager [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Terminating instance
Apr 14 03:41:43 user nova-compute[71323]: DEBUG nova.compute.manager [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Start destroying the instance on the hypervisor. {{(pid=71323) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3105}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-unplugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] No waiting events found dispatching network-vif-unplugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG nova.compute.manager [req-7aa1cb93-e756-4ba4-883b-40aa7b89a91e req-7eb6f238-2c93-4e3d-91f4-d28a55fbc02a service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-unplugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 for instance with task_state deleting. {{(pid=71323) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10760}}
Apr 14 03:41:43 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Instance destroyed successfully.
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.objects.instance [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lazy-loading 'resources' on Instance uuid 536a066d-074b-4a88-94d2-9815421b70da {{(pid=71323) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.virt.libvirt.vif [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2023-04-14T03:39:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-712565026',display_name='tempest-VolumesActionsTest-instance-712565026',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-712565026',id=47,image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-04-14T03:39:52Z,launched_on='user',locked=False,locked_by=None,memory_mb=128,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='778eb822f6cb4f9c9c388ee7ed8a2071',ramdisk_id='',reservation_id='r-f1xlak59',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='98e6a247-250b-4bb5-a947-e7f93598d16e',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.5.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-376578855',owner_user_name='tempest-VolumesActionsTest-376578855-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-04-14T03:39:53Z,user_data=None,user_id='2975e8c90d3e41dfa099c98f41635950',uuid=536a066d-074b-4a88-94d2-9815421b70da,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converting VIF {"id": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "address": "fa:16:3e:f6:c7:c0", "network": {"id": "fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6", "bridge": "br-int", "label": "tempest-VolumesActionsTest-433798697-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "778eb822f6cb4f9c9c388ee7ed8a2071", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8c7a028d-cb", "ovs_interfaceid": "8c7a028d-cb6f-4cc7-8469-07fb8c5afe17", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.network.os_vif_util [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb') {{(pid=71323) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG os_vif [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb') {{(pid=71323) unplug /usr/local/lib/python3.10/dist-packages/os_vif/__init__.py:109}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 22 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8c7a028d-cb, bridge=br-int, if_exists=True) {{(pid=71323) do_commit /usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:44 user nova-compute[71323]: INFO os_vif [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:f6:c7:c0,bridge_name='br-int',has_traffic_filtering=True,id=8c7a028d-cb6f-4cc7-8469-07fb8c5afe17,network=Network(fbf5eb1c-30a9-4e33-a4da-ef1fef9c24f6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8c7a028d-cb')
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Deleting instance files /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da_del
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.virt.libvirt.driver [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Deletion of /opt/stack/data/nova/instances/536a066d-074b-4a88-94d2-9815421b70da_del complete
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.compute.manager [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Took 0.69 seconds to destroy the instance on the hypervisor.
Apr 14 03:41:44 user nova-compute[71323]: DEBUG oslo.service.loopingcall [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=71323) func /usr/local/lib/python3.10/dist-packages/oslo_service/loopingcall.py:435}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.compute.manager [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Deallocating network for instance {{(pid=71323) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2264}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] deallocate_for_instance() {{(pid=71323) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1793}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.network.neutron [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Updating instance_info_cache with network_info: [] {{(pid=71323) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Took 0.57 seconds to deallocate network for instance.
Apr 14 03:41:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:41:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.116s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:44 user nova-compute[71323]: INFO nova.scheduler.client.report [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Deleted allocations for instance 536a066d-074b-4a88-94d2-9815421b70da
Apr 14 03:41:44 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-87fd877c-bc0b-467b-b486-72dfdfd33225 tempest-VolumesActionsTest-376578855 tempest-VolumesActionsTest-376578855-project-member] Lock "536a066d-074b-4a88-94d2-9815421b70da" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.543s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:41:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] Acquiring lock "536a066d-074b-4a88-94d2-9815421b70da-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:41:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:41:45 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] Lock "536a066d-074b-4a88-94d2-9815421b70da-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:41:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] No waiting events found dispatching network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Apr 14 03:41:45 user nova-compute[71323]: WARNING nova.compute.manager [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received unexpected event network-vif-plugged-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 for instance with vm_state deleted and task_state None.
Apr 14 03:41:45 user nova-compute[71323]: DEBUG nova.compute.manager [req-be523709-2b39-4678-817e-54163471f79c req-5c1d1019-a7a6-45db-8f87-bb42b6a4f2c3 service nova] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Received event network-vif-deleted-8c7a028d-cb6f-4cc7-8469-07fb8c5afe17 {{(pid=71323) external_instance_event /opt/stack/nova/nova/compute/manager.py:10982}}
Apr 14 03:41:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1681443704.0026255, 536a066d-074b-4a88-94d2-9815421b70da => Stopped> {{(pid=71323) emit_event /opt/stack/nova/nova/virt/driver.py:1653}}
Apr 14 03:41:59 user nova-compute[71323]: INFO nova.compute.manager [-] [instance: 536a066d-074b-4a88-94d2-9815421b70da] VM Stopped (Lifecycle Event)
Apr 14 03:41:59 user nova-compute[71323]: DEBUG nova.compute.manager [None req-7050a530-ad8e-4eab-afe1-1ad9917dc000 None None] [instance: 536a066d-074b-4a88-94d2-9815421b70da] Checking state {{(pid=71323) _get_power_state /opt/stack/nova/nova/compute/manager.py:1762}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:41:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:20 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:20 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:42:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:27 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:42:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:42:28 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:42:28 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:29 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:42:29 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8906MB free_disk=26.28929901123047GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:42:29 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.133s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:42:31 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:31 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:34 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:42:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:42:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:42:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:42 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:42:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:02 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:03 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:16 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:22 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:22 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:43:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:43:30 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:43:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:43:31 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=8896MB free_disk=26.279136657714844GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:43:31 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.147s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:43:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:35 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Starting heal instance info cache {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9792}}
Apr 14 03:43:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Rebuilding the list of instances to heal {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9796}}
Apr 14 03:43:37 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Didn't find any instances for network info cache update. {{(pid=71323) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9878}}
Apr 14 03:43:37 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:43:39 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:44 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:49 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:54 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:43:59 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:44:04 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:09 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:14 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:44:18 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:19 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:44:23 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:23 user nova-compute[71323]: DEBUG nova.compute.manager [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=71323) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10411}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:44:24 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:44:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:28 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._sync_power_states {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:29 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:263}}
Apr 14 03:44:30 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager.update_available_resource {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:44:32 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Auditing locally available compute resources for user (node: user) {{(pid=71323) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:861}}
Apr 14 03:44:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:44:33 user nova-compute[71323]: WARNING nova.virt.libvirt.driver [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Hypervisor/Node resource view: name=user free_ram=9003MB free_disk=26.260486602783203GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=71323) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1034}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1057}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Final resource view: name=user phys_ram=16023MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=71323) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1066}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.compute.provider_tree [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed in ProviderTree for provider: 8eb2236f-d4b5-4afe-871b-37575055ded0 {{(pid=71323) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.scheduler.client.report [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Inventory has not changed for provider 8eb2236f-d4b5-4afe-871b-37575055ded0 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16023, 'reserved': 512, 'min_unit': 1, 'max_unit': 16023, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=71323) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG nova.compute.resource_tracker [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Compute_service record updated for user:user {{(pid=71323) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:995}}
Apr 14 03:44:33 user nova-compute[71323]: DEBUG oslo_concurrency.lockutils [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.411s {{(pid=71323) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=71323) run /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:117}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=71323) _transition /usr/local/lib/python3.10/dist-packages/ovs/reconnect.py:519}}
Apr 14 03:44:34 user nova-compute[71323]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=71323) __log_wakeup /usr/local/lib/python3.10/dist-packages/ovs/poller.py:248}}
Apr 14 03:44:36 user nova-compute[71323]: DEBUG oslo_service.periodic_task [None req-afa55d57-7a22-4a9a-9199-77bfaea5139b None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=71323) run_periodic_tasks /usr/local/lib/python3.10/dist-packages/oslo_service/periodic_task.py:210}}